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FOREWORD 


In  1980  the  Assistant  Secretary  of  Defense  directed  all  services  to 
pursue  a  long-range  systematic  program  to  validate  the  Armed  Services  Voca¬ 
tional  Aptitude  Battery  (ASVAB)  and  to  compare  enlistment  standards  with 
on-the-job  performance.  The  Army  has  been  investigating  the  validity  of  the 
ASVAB,  as  well  as  several  new  predictor  measures,  for  a  sample  of  20  diverse 
Military  Occupation  Specialties  (MOS),  This  effort,  known  as  Project  A,  has 
been  very  successful  in  validating  the  ASVAB,  as  well  as  in  providing  the 
Army  with  a  greater  understanding  of  knowledge,  skills,  abilities,  and  other 
personal  characteristics  (KSAOs)  required  for  these  20  MOS. 

A  major  question  now  facing  the  Army  is  how  to  extend  the  wealth  of 
data  collected  for  Project  A  to  the  other  250-plus  entry-level  Army  MOS  and 
to  new  MOS  created  for  new  hardware  systems  as  they  become  operational.  A 
second  challenge  is  to  determine  the  methods  needed  for  setting  job  per¬ 
formance  standards  that  can  be  used  in  making  selection  and  classification 
decisions. 

The  Army  currently  has  a  research  project,  the  Synthetic  Validity 
Project  (SYNVAL),  that  addresses  these  challenges.  Specifically,  the 
objectives  of  SYNVAL  are  to  (1)  evaluate  synthetic  validity  techniques  for 
determining  MOS-specific  selection  composites  for  each  MOS;  and  (2)  evaluate 
alternative  methods  for  setting  minimum  qualifying  scores  on  each  of  these 
composites.  The  research  will  proceed  in  three  phases.  Phase  I  was  re¬ 
cently  completed  and  this  document  provides  information  on  Phase  I  research 
plans,  objectives,  and  results. 

Based  on  the  results  of  the  evaluations,  recommendations  will  be  made 
for  the  most  promising  approach  for  developing  job  performance  prediction 
equations  for  all  of  the  Army's  250-plus  MOS  and  for  setting  performance 
standards  for  these  MOS.  The  technical  quality  of  this  project  is  guided  by 
the  Scientific  Advisory  Committee,  Drs.  Phil  Bobko  (Chair),  Robert  Linn, 
Richard  Jaeger,  Joyce  Shields,  and  Robert  Guion. 
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ARMY  SYNTHETIC  VALIDITY  PROJECT:  REPORT  OF  PHASE  I  RESULTS 


EXECUTIVE  SUMMARY 


Requirement; 

The  two  major  objectives  of  the  Army  Synthetic  Validity  Project  are  to 
identify  and  evaluate  procedures  for 

•  identifying  an  optimal  composite  of  selection  measures  for  any 
Army  enlisted  Military  Occupational  Specialty  (MOS)  and  estimating 
the  validity  of  this  composite  for  predicting  job  performance;  and 

•  setting  a  minimum  qualifying  score  so  as  to  assure  a  reasonable 
probability  of  successful  job  performance,  as  well  as  other  ap¬ 
propriate  cutting  scores  for  other  critical  selection  decisions 
(e.g.,  for  selecting  recruits  with  potential  for  outstanding 
performance). 

Synthetic  validity  approaches  typically  begin  with  the  identification 
of  a  set  of  job  components  that  can  be  used  to  describe  the  population  of 
jobs  being  studied.  A  prediction  equation  is  derived  for  linking  available 
selection  tests  to  each  component.  Subject  matter  experts  (SMEs)  are  asked 
to  identify  the  importance  of  each  component  to  overall  job  performance. 
Finally,  the  prediction  equations  for  the  various  components  are  weighted 
according  to  the  importance  judgment  weights  and  summed  to  obtain  an  equa¬ 
tion  for  predicting  overall  performance  for  the  job. 

The  standard  setting  task  of  the  Synthetic  Validity  Project  is  charged 
with  developing  procedures  for  specifying  minimum  qualifying  scores  and 
other  appropriate  cut  scores  on  the  predictor  composites  identified  for  each 
job.  Procedures  will  be  developed  for  identifying  job  performance  standards 
for  each  job,  and  these  performance  standards  will  then  be  linked  to  scores 
on  the  predictor  composite  for  that  job. 


Procedure: 

There  are  three  research  phases  in  the  Project.  In  each  phase,  syn¬ 
thetic  validity  procedures  and  standard  setting  procedures  are  developed  or 
refined  and  then  tried  out  on  a  new  sample  of  MOS.  A  major  goal  in  Phase  I 
for  synthetic  validity  was  to  obtain  and  evaluate  synthetic  prediction 
equations  for  three  MOS.  Three  job  component  models  (consisting  of  tasks, 
activities,  or  attributes)  were  developed  and  used  to  obtain  job  description 
judgments.  Predictors  were  linked  via  expert  judgment  to  the  job  compo¬ 
nents.  Various  ways  of  generating  prediction  equations  were  investigated. 

A  second  goal  was  to  evaluate  difference  in  the  job  descriptions  generated 
by  different  types  of  judges. 


A  major  goal  in  Phase  I  standard  setting  was  to  investigate  different 
ways  of  setting  performance  standards.  Performance  level  definitions  were 
developed.  Three  standard  setting  methods  reflecting  performance  on  tasks, 
critical  incidents,  and^by  soldiers  were  developed  to  obtain  component 
standards.  One  method  was  developed  for  combining  the  component  standards. 


Findings: 

For  synthetic  validity,  the  completion  of  Phase  I  represents  a  major 
accomplishment  for  the  project.  First,  each  of  the  three  job  component 
models  produced  reliable  and  comprehensive  job  descriptions.  Second,  valid 
predictors  were  successfully  linked  via  expert  judgment  to  those  job  com¬ 
ponents.  Third,  using  job  description  and  job  component  validity 
information,  we  formed  prediction  equations  that  had  high  predictive 
validity  for  each  of  the  three  jobs. 

For  standard  setting.  Army  SMEs  found  the  performance  level  definitions 
to  be  reasonable  and  workable.  The  three  methods  for  setting  standards  re¬ 
sulted  in  different  standards  and  also  in  some  differences  in  the  degree  of 
consensus  among  judges  in  setting  the  standards.  In  deriving  an  overall 
standard  from  component  standards,  there  was  evidence  that  a  linear  com¬ 
pensatory  model  accurately  captures  the  judges'  aggregation  strategies. 


Utilization  of  Findings: 

At  the  conclusion  of  Phase  I,  we  have  shown  that  synthetic  validity 
yields  valid  predictions  for  three  jobs.  Future  phases  of  the  project  will 
aim  to  extend  the  validity.  In  Phase  II,  we  will  continue  work  to  refine 
the  job  component  models,  perhaps  to  reduce  the  number  of  models  and  explore 
an  optimal  way  of  generating  prediction  equations  to  yield  the  highest 
predictability  and  discriminability  among  jobs. 

Meaningful  performance  standards  were  obtained  for  the  three  jobs. 
Future  phases  of  the  project  will  refine  the  methods  to  yield  better 
agreement  among  the  judges  and  greater  convergence  across  methods. 
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CHAPTER  1: 


ARMY  SYNTHETIC  VALIDITY  PROJECT;  REPORT  OF  PHASE  I  RESULTS 


INTRODUCTION 

The  Army  Synthetic  Validity  (SYNVAL)  Project  is  designed  to 
evaluate  techniques  for  determining  critical  entry  requirements  for 
each  enlisted  Military  Occupational  Specialty  (MOS).  The  project  has 
two  main  components.  The  first  component  is  an  evaluation  of 
synthetic  validation  (as  defined  by  Lawshe,  1952;  Balma,  1959)  as  a 
means  for  identifying  and  weighting  the  critical  knowledge,  skills, 
abilities,  and  other  personal  characteristics  (KSAOs)  required  for 
successful  job  performance.  These  weighted  KSAOs  are  then  used  to 
create  an  appropriate  test  battery  for  selection.  The  second 
component  is  the  development  and  refinement  of  procedures  for  setting 
standards  for  on-the-job  performance  in  each  MOS  and  for  linking 
selection  composite  score  levels  to  these  job  performance  standards. 

The  SYNVAL  project  is  organized  into  three  phases.  In  each 
phase,  synthetic  validation  procedures  and  standard  setting 
procedures  are  developed  or  refined  and  then  tried  out  on  a  new 
sample  of  MOS.  This  report  documents  the  results  of  Phase  I  data 
collection  and  analysis  activities. 


Summary  of  Previous  Reports 

Considerable  work  preceded  the  Phase  I  data  collection.  Much  of 
this  work  is  reported  in  detail  in  other  documents  and  is  only 
briefly  mentioned  here.  We  began  the  project  with  a  review  of 
literature  on  synthetic  validation  procedures  and  prior  efforts  to 
iralidate  these  procedures.  A  significant  part  of  the  review  also 
covered  various  job  component  models  that  have  been  used  in  job 
analytic  research  and  in  synthetic  validation.  The  results  of  this 
review  are  reported  in  A  Review  of  Models  and  Procedures  for 
Synthetic  Validation  for  Entrv-level  Army  Jobs  (Crafts,  Szenas,  Chia, 
&  Pulakos,  1988). 

Synthetic  validation  begins  with  the  development  of  a  set  of  job 
descriptors  that  can  be  used  to  characterize  any  job  in  the  domain 
under  consideration.  We  call  these  descriptors  "components"  and 
refer  to  the  model  driving  the  specification  of  these  descriptors  as 
a  Job  Component  Model.  As  described  in  the  sections  that  follow,  we 
developed  three  alternative  job  component  models  and  pilot  tested 
instruments  for  obtaining  job  descriptions  using  each  model.  The 
results  of  this  pilot  test  are  reported  in  Synthetic  Validation 
Project:  Pilot  Test  Report  (Chia,  Owens-Kurtz,  Peterson,  &  Szenas, 
1988) . 

Finally,  we  also  reviewed  literature  relating  to  setting  perfor¬ 
mance  standards.  The  results  of  this  review  are  reported  in  A  Review 
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Finally,  we  also  reviewed  literature  relating  to  setting  perfor¬ 
mance  standards.  The  results  of  this  review  are  reported  in  A  Review 
of  Procedures  for  Setting  Job  Performance  Standards  (Pulakos,  Wise, 
Arabian,  Heon,  &  Delaplane,  1989). 

Synthetic  Validation  and  Phase  I  Objectives 

The  overall  goal  of  the  Synthetic  Validation  Project  is  to  develop 
and  evaluate  alternative  synthetic  procedures  for  choosing  valid 
selection  and  classification  predictor  measures  for  specific  jobs.  The 
project  is  a  logical  extension  of  the  Army's  landmark  research  in 
selection  and  classification  (Projects  A  and  B).  Chapter  2  provides 
the  background  for  the  SYNVAL  project  and  also  describes  the  overall 
design  and  plan  of  the  project  for  generalizing  the  validity 
information  from  a  limited  set  of  jobs  to  all  jobs.  The  project  has 
adopted  a  phased  approach  for  extending  validity  to  all  jobs. 

A  major  goal  for  synthetic  validation  for  Phase  I  was  to  obtain 
synthetic  prediction  equations  for  three  M0S-- Infantryman  (IIB),  Light- 
wheel  Vehicle  Mechanic  (63B),  and  Administrative  Specialist  (71L).  For 
that  to  occur,  two  preparatory  steps  must  be  satisfied.  First,  we  must 
be  able  to  describe  an  MOS  reliably  in  terms  of  job  components.  A  job 
component  model  for  this  purpose  should  yield  a  reliable  and 
comprehensive  description  of  a  job.  Second,  after  the  job  content  is 
specified,  valid  predictors  must  be  linked  to  those  job  components.  In 
Phase  I,  we  investigated  using  job  experts  and  testing  experts  to 
estimate  the  validity  of  a  host  of  predictor  constructs  for  job 
components  and  for  more  macro  areas  of  job  performance.  These  two 
steps  provided  all  the  needed  information  to  generate  synthetic 
equations  for  three  jobs.  Clearly,  these  procedures  require  a  major 
role  from  job  experts  for  providing  judgments  about  job  content  and 
performance.  Therefore,  another  goal  for  Phase  I  was  to  examine  how 
judges  should  be  selected. 

Job  component  models.  Three  job  component  models  were 
investigated  in  Phase  I.  The  first  job  component  model  is  called  the 
"Task  Category  Model."  It  is  composed  of  major  job  tasks  that 
describe  the  content  of  jobs.  The  second  job  component  model,  "Job 
Activity  Model,"  is  composed  of  general  job  behaviors  that  may  be 
relevant  for  several  specific  jobs.  Judges  are  asked  to  describe  a  job 
in  terms  of  critical  components.  Chapter  3  is  a  description  of  the 
development  of  the  two  models  and  of  results  of  using  the  two  models 
with  Army  job  experts.  The  relevant  Phase  I  issues  are: 
reliability  of  criticality  judgments  (i.e.,  frequency  and  importance), 
discriminabil ity  between  jobs,  comprehensiveness  of  coverage,  and 
appropriateness  of  judges. 

The  third  job  component  model,  the  "Attribute  Model,"  is  composed 
of  predictor  variables  or  constructs  such  as  cognitive  abilities,  non- 
cognitive  attributes  such  as  temperament  and  interests,  and  psychomotor 
and  perceptual  abilities.  A  job  can  be  described  directly  in  terms  of 
these  attribute  requirements  by  having  job  experts  estimate  the 
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analyses.  Therefore,  the  description  of  the  development  of  the 
attribute  model  and  results  of  analyzing  validity  judgments  are 
documented  separately  in  Chapter  4. 

Linking  predictor  constructs  to  iob  components  and  iob 
performance.  One  of  the  more  difficult  steps  in  synthetic  approaches 
to  validation  has  been  to  obtain  validity  information  for  the  job 
components.  However,  recent  research  by  Wing,  Peterson,  and  Hoffman 
(1984)  and  Schmidt,  Hunter,  Croll,  and  McKenzie  (1983)  suggested  that 
expert  judgments  of  validity  are  highly  accurate  estimates  of 
validities  obtained  empirically.  Therefore  in  Phase  I,  we  also 
collected  validity  estimates  for  31  predictors  for  job  components  and 
for  more  macro  levels  of  performance.  These  procedures  and  results  are 
presented  in  Chapter  5. 

Evaluation  of  synthetic  equations.  Chapter  6  describes  how 
synthetic  equations  are  formed  for  each  job.  There  can  be  a  large 
number  of  permutations  for  forming  prediction  equations  from  the 
different  job  component  models  and  weighting  schemes.  Some 
combinations  resulted  in  better  predictability  while  others  resulted  in 
better  differential  prediction  among  jobs.  Chapter  6  begins  to  explore 
some  of  those  possibilities. 

Judge  characteristics  and  selection.  During  pilot  tests  conducted 
prior  to  Phase  I,  we  requested  Army  officers,  senior  NCOs,  and 
civilians  to  provide  the  job  description  judgments.  We  have  found  that 
as  a  group,  the  participants  have  provided  reasonably  reliable  and 
useful  data.  With  the  same  types  of  judges  in  Phase  I,  we  began  to 
examine  the  individual  rater  characteristics  that  led  to  more  accurate 
judgments.  This  research  is  reported  in  Chapter  7.  The  results  will 
have  implications  for  judge  selection  or  screening  in  future  research 
phases  and  when  the  procedures  are  operational. 

Standard  Setting  and  Phase  I  Objectives 

The  standard  setting  component  of  the  Synthetic  Validation  Project 
is  charged  with  developing  procedures  for  specifying  minimum  qualifying 
scores  and  other  appropriate  cut  scores  on  the  predictor  composites 
identified  for  each  job.  We  are  evaluating  a  two-step  approach  to 
standard  setting.  For  the  first  step,  we  intend  to  develop  procedures 
for  identifying  appropriate  job  performance  standards  for  each  MOS. 

For  the  second  step  we  will  specify  a  method  for  linking  these 
performance  standards  to  scores  on  the  predictor  composite  for  that 
job. 


Phase  I  research  began  to  address  the  first  step.  There  were 
three  subgoals.  First,  performance  standards  should  be  defined.  We 
began  by  defining  four  levels  of  acceptability  as  an  alternative  to  the 
more  common  "all  or  none"  (acceptable  or  unacceptable)  approach.  We 
tested  these  definitions  with  Army  SMEs.  Second,  we  attempted  to 
locate  these  performance  standards  (acceptability  levels)  onto  a 
performance  continuum.  We  tested  three  methods  for  obtain  performance 
level  descriptions  for  different  performance  areas.  Third,  the 
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performance  standards  for  the  different  areas  should  be  combined.  We 
tested  one  method  for  obtaining  overall  performance  standards. 


Performance  level  definitions.  In  Phase  I,  we  proposed  four 
levels  of  performance.  These  four  levels  were  as  follows: 


unacceptable  -  Soldiers  who  consistently  perform  like  this  do  not 
belong  in  the  Army.  Their  performance  is  hurting 
the  Army,  and  they  should  be  discharged  early. 

marginal  *  Soldiers  who  consistently  perform  like  this  need 
remedial  training.  Their  performance  is  of  little 
or  no  benefit  to  the  Army.  Unless  they  receive 
training  and  improve  their  performance,  they  should 
be  barred  from  re-enlistment. 


acceptable  *  Soldiers  who  consistently  perform  like  this  are 
doing  an  adequate  Job.  They  are  making  positive 
contributions  to  the  Army.  They  should  be  allowed 
to  re-enlist. 


outstanding  «  Soldiers  who  consistently  perform  like  this  are 

doing  extremely  well.  They  are  making  exceptional 
contributions  to  the  Array  and  are  excellent 
examples  to  their  peers.  They  should  be  encouraged 
to  re-enlist  and  should  be  given  special 
consideration  for  promotion  or  extra 
responsibilities. 


The  four  levels  of  acceptability  of  a  soldier's  performance  were 
intended  to  correspond  to  operational  responses  to  such  performance. 
One  of  the  goals  in  Phase  I  was  to  clarify  the  definition  of  the  four 
levels.  Army  SMEs  reviewed  these  performance  levels  and  attempted  to 
use  them  for  performance  level  descriptions. 


Performance  level  descriptions.  Another  goal  for  Phase  I  was  to 
define  performance  in  terms  of  the  four  levels.  Three  methods  for 
locating  the  levels  of  performance  were  tested,  and  these  three  methods 
were  designed  to  allowed  us  to  determine  soldiers'  performance  level  as 
measured  in  Project  A.  Project  A  research  (Campbell,  1986}  indicated 
that  a  soldier's  Job  performance  is  multi-dimensional.  Therefore,  we 
attempted  to  set  standards  for  several  relevant  Job  performance 
dimensions.  These  dimensions  included  General  Soldiering  Proficiency, 
Effort  and  Leadership,  Personal  Discipline,  and  Physical  Fitness  and 
Military  Bearing,  and  one  to  three  performance  areas  that  are  specific 
to  the  three  Phase  I  Jobs. 

The  first  approach  was  the  Soldier-based  method.  This  method 
required  Judges  to  estimate  directly  the  percent  of  incumbents  who  are 
performing  unacceptably,  marginally,  acceptably,  or  outstandingly  on 
each  performance  dimension.  The  second  approach  was  based  on  examples 
of  Army-wide  and  MOS-specific  critical  incidents  gathered  for  Project 
A.  Judges  were  asked  to  rate  whether  an  incident  was  unacceptable, 
marginal,  acceptable,  or  outstanding.  The  third  approach  was  the  Task- 
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based  method.  We  presented  ten  levels  of  performance  using  Project  A 
MOS-specific  hands-on  test  data  and  asked  judges  to  rate  the 
acceptability  of  each  performance  level. 


Some  of  the  key  issues  that  we  addressed  include  how  much 
agreement  was  there  among  judges  and  how  comparable  were  the  standards 
between  the  standard  setting  methods.  In  addition,  we  also  examined 
the  effects  of  two  manipulations  on  the  resulting  standards.  One 
manipulation  was  the  availability  of  normative  data.  Half  of  the 
judges  received  normative  data  and  half  did  not.  The  other 
manipulation  was  practice.  Again,  half  of  the  judges  was  allowed  to 
work  on  a  short  practice  exercise  before  commencing  on  the  actual 
exercise.  The  resulting  standards  for  the  three  approaches  and  the 
effects  of  the  manipulations  are  presented  in  Chapter  7. 

Deriving  overall  standards.  The  above  three  standard  setting 
approaches  were  intended  to  obtain  performance  levels  on  performance 
dimensions.  In  order  to  characterize  a  more  global  performance  level 
for  the  job,  the  performance  levels  on  the  performance  dimensions  can 
be  combined.  In  Phase  I,  we  explored  how  judges  aggregated  or  combined 
the  component  standards  into  an  overall  standard.  We  constructed  forty 
hypothetical  soldiers  with  varying  levels  of  performance  on  three 
performance  dimensions  and  asked  judges  to  describe  the  acceptability 
of  each  soldier's  overall  performance.  Some  questions  that  were 
explored  included  the  strategies  for  combining  component  standards  and 
how  judges  viewed  extreme  performance.  These  results  are  described  in 
Chapter  8, 

Review  bv  Scientific  Advisory  Committee 

Part  of  the  project  plan  includes  the  review  of  plans  and  research 
results  by  the  Scientific  Advisory  Committee  (SAC).  Members  of  the  SAC 
are  Drs.  Phil  Bobko,  Robert  Guion,  Robert  Linn,  Richard  Jaeger,  and 
Joyce  Shields.  The  Phase  I  results  reported  in  this  document  have  been 
presented  to  and  reviewed  by  the  SAC  on  November  9  and  10,  1988.  A 
copy  of  the  minutes  of  that  meeting  is  found  in  Appendix  A.  During  the 
meeting,  SAC  members  offered  a  number  of  specific  suggestions  for 
future  research.  These  suggestions  and  their  implications  are 
summarized  in  the  last  chapter. 
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The  Application  of  Synthetic  Validation  to  the  Development 
of  System  Wide  Selection  and  Classification  Procedures 

John  P.  Campbell 
University  of  Minnesota 

Lauress  L.  Wise 

American  Institutes  for  Research 

Jane  Arabian 
Army  Research  Institute 


SUMMARY 

The  overall  goal  of  the  Synthetic  Validation  Project  is  to  develop 
and  evaluate  alternative  synthetic  procedures  for  choosing  valid 
selection  and  classification  predictor  measures  for  specific  jobs. 

The  U.S.  Army  selects  110,000  -  130,000  people  per  year  from 
400,000  -  500,000  applicants  and  must  assign  them  to  275  different 
entry  level  jobs  (MOS)  while  meeting  a  number  of  constraints.  It  is 
a  formidable  individual  and  organizational  decision-making  task  and 
makes  heavy  demands  on  personnel  research. 

The  ongoing  Army  Selection  and  Classification  Project  (Project 
A)  has  produced  a  large  selection/classification  research  data  bank 
by  choosing  a  sample  of  19  jobs  from  the  population  of  275  jobs  and 
assessing  300-600  people  from  each  job  on  a  4-hour  experimental 
predictor  battery  and  12  hours  of  criterion  measurement.  However, 
there  Is  still  a  problem  of  how  to  develop  selection  and  classification 
procedures  for  the  256  remaining  jobs.  Three  major  options  are  to: 
conduct  additional  criterion-related  research,  cluster  jobs  on  the 
basis  of  their  judged  similarity  around  the  19  empirically  based 
prediction  equations  (i.e.  validity  generalizations),  or  use  synthetic 
validation  procedures  to  selection  and  weight  predictors  in  a  battery. 

The  synthetic  validation  project  Is  designed  to  evaluate  the 
third  strategy  by  developing  three  alternative  synthetic  procedures, 
extensively  pilot  testing  each  one,  and  then  evaluating  them  against 
each  other  and  against  the  empirically  derived  prediction  equations 
that  are  part  of  the  Project  A  database. 

This  chapter  discusses  the  nature  of  the  selection/classification 
problem  in  the  Army,  the  parameters  of  the  Project  A  data  file,  and 
the  overall  design  of  the  current  project.  The  parameters  that 
distinguish  the  alternative  synthetic  procedures  being  compared  and 
evaluated  are  the  unit  of  analysis,  the  type  of  expert  judge,  and  the 
type  of  judgment  required.  The  relevant  data  are  the  distributional 
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properties  and  reliabilities  of  the  judgments,  the  discriminabil ity 
among  jobs  produced  by  each  method,  and  the  correspondence  between 
synthetic  and  empirical  estimates  in  the  Project  A  validation  sample 
of  19  jobs  (MOS).  The  remaining  chapters  look  at  the  major  parts  of 
the  project  in  more  detail. 
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CHAPTER  2: 


The  Application  of  Synthetic  Validation  to  the  Development 
of  System  Wide  Selection  and  Classification  Procedures 


THE  ARMY  CONTEXT 

The  Army  faces  the  following  decision-making  requirements,  as 
regards  the  selection  and  classification  of  entry  level  enlisted 
personnel  (new  accessions).  During  the  past  5-10  years  approximately 
400,000  -  500,000  people  have  applied  each  year  for  approximately 
110,000  -  130,000  openings.  The  available  openings  are  distributed 
unevenly  across  approximately  275  different  jobs  ranging  from 
infantryman  to  helicopter  engineer  to  mechanic  to  paramedic  to 
administrative/clerical  specialist.  Each  new  accession  goes 
immediately  to  basic  training  and  then  to  advanced  training  in 
his/her  chosen  specialty.  The  number  of  training  slots  that  will  be 
available  is  budgeted  at  least  one  year  in  advance  and  many 
cost/benefit  parameters  are  optimized  if  every  seat  is  filled  with 
appropriate  people  on  the  day  the  class  starts.  The  individual 
choice  of  MOS  is  a  function  of  training  seat  availability  at  a 
particular  time,  the  current  priority  for  "filling"  the  MOS,  the 
individual's  preference,  and  whether  or  not  the  individual's  scores 
on  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  meet 
certain  cutoffs.  This  is  a  complex  decision  process  which  must  take 
place  very  quickly  and  is  made  on  the  basis  of  a  relatively  small 
amount  of  information. 

External  issues  about  which  the  Army  must  be  concerned  are  a 
fluctuating  labor  supply  with  its  general  downward  trend,  and  the  ups 
and  downs  of  the  federal  budget  which  have  a  direct  effect  on 
resources  devoted  to  recruiting  and  the  resulting  nature  of  the 
applicant  pool.  At  the  same  time,  new  equipment  and  new  systems  have 
been  developed  and  the  technical  content  and  ability  requirements  of 
almost  all  MOS  have  increased  markedly. 

As  a  consequence  of  all  of  the  above,  accurate  selection  and 
optimal  classification  have  become  more  critical  than  ever.  At  the 
same  time,  there  is  constant  pressure  on  all  the  defense  services  to 
provide  evidence  that  their  personnel  decision-making  procedures  are 
appropriate  and  valid.  As  an  organization,  the  Army  is  a  very  large 
and  very  visible  employer. 


PROJECTS  A  i  B 

The  Synthetic  Validation  Project  is  functionally  related  to  two 
other  research  and  development  projects  aimed  at  improving  the  Army's 
selection  and  classification  decision  making  procedures:  Project  A 
and  Project  B. 
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Project  B 

Project  B  is  based  on  theory  and  method  in  econometrics  and 
operations  research.  It  has  developed  the  models  and  software  for  an 
enlisted  personnel  assignment  system  that  takes  into  account: 

•  forecasts  in  the  future  applicant  supply 

•  forecasts  of  personnel  needs  in  each  MOS 

•  hiring  goals  for  different  subpopulations 

•  the  rate  at  which  training  class  slots  are  currently  filling 

•  the  MOS  priorities  designated  by  the  Army 

•  the  differential  utility  of  different  expected  levels  of 
performance  within  and  across  MOS 

•  the  level  of  selection  accuracy  and  differential  prediction 
across  MOS  provided  by  the  predictor  battery. 

Project  B  is  intended  to  be  a  state-of'the-art  algorithm  for  optimizing 
personnel  decisions,  given  certain  goals,  and  for  conducting  a  wide 
variety  of  "what  if"  exercises  as  regards  changes  in  labor  supply, 
priorities,  utilities,  and  criterion  content. 

Project  A 

Project  A  is  a  very  large  personnel  selection  and  classification 
validation  project  that  was  intended  to  use  a  sample  of  jobs  (MOS) 
from  the  entire  population  of  enlisted  MOS  to  validate  both  the 
existing  test  battery  (ASVAB)  and  a  battery  of  newly  developed 
selection/classification  tests  against  a  comprehensive  set  of 
performance  measures.  The  major  research  issues  revolved  around: 

•  how  to  define  and  measure  job  performance 

§  the  tradeoff  between  the  number  of  jobs  vs.  the  sample  size 
for  each  job,  given  that  resources  did  not  permit  drawing  a 
sample  from  each  of  the  275  MOS 

•  identification  of  predictor  domains  with  the  highest 
potential  for  adding  selection  validity  and  classification 
validity  to  the  existing  ASVAB 

•  how  specific  variables  should  be  targeted  in  each  domain 
for  predictor  development 

•  how  performance  measures  should  be  aggregated  into  composites 
for  validation  purposes 
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•  how  to  choose  predictor  batteries  and  estimate  validity  for 
jobs  (MOS)  for  which  no  empirical  data  could  be  obtained. 

Design  and  Method 


To  pursue  the  project's  objectives  while  addressing  the  above 
issues,  the  following  design  was  used  in  Project  A, 

There  were  two  major  validation  samples:  a  concurrent  sample 
taken  from  the  1983/84  cohort  and  measured  on  both  the  new  predictors 
and  new  criterion  measures  in  1985;  and  a  longitudinal  sample 
assessed  on  the  predictors  when  they  entered  the  Army  in  1986/87  and 
tested  on  the  performance  measures  in  1988/89.  Each  sample  consists 
of  80  to  over  900  people  in  each  of  21  MOS  which  were  selected  to  be 
representative  of  the  entire  population  of  enlisted  MOS. 

Consequently,  both  samples  contain  over  20,000  individuals  with 
predictor  and  criterion  measures. 

Criterion  measures  were  developed  by  conducting  an  extensive 
task  analysis  and  critical  incident  analysis  of  each  MOS.  All 
available  sources  and  multiple  expert  reviews  were  used  to  generate  a 
full  listing  of  all  tasks  in  each  MOS  as  well  as  judgments  about  the 
criticality  and  difficulty  of  each  task  and  the  similarity  among 
tasks.  The  critical  incident  analysis  produced  a  complete  set  of 
performance  dimensions  for  each  MOS.  For  a  representative  sample  of 
critical  tasks  in  each  MOS,  job  sample  (hands-on)  exercises,  paper- 
and-pencil  knowledge  tests,  and  rating  scales  were  developed.  Also, 
behavioral  rating  scales  were  developed  for  each  of  the  dimensions 
that  survived  the  critical  incident  retranslation  and  SME  reviews. 

In  addition,  rating  scales  were  developed  to  assess  expected 
performance  in  combat.  Finally,  existing  administrative  records  were 
examined  and  six  variables  retained  as  performance  indicators  (e.g., 
number  of  awards  and  letters  of  commendation).  The  full  performance 
assessment  required  12  hours  per  individual. 


Potential  new  predictor  variables  were  selected  through  a 
painstaking  process  of  literature  search,  expert  review,  and 
evaluation  of  previous  research.  The  goal  was  to  produce  a  four-hour 
battery  of  new  tests  that  would  maximize  the  chances  of  improving 
selection/classification  accuracy  for  the  entire  system  (i.e. 
population  of  MOS).  In  the  end,  the  domains  from  which  the 
experimental  predictors  were  sampled  were  the  following  (in  addition 
to  the  ASVAB) 


t  spatial  ability 

•  perceptual  speed  and  accuracy 

•  psychomotor  abilities 

•  personal i ty/temperament 

•  vocational  Interests 

•  biographical  history 


j 
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The  major  steps  in  the  analysis  were  directed  first  at  developing 
a  basic  set  of  predictor  scores  from  the  four-hour  battery,  a  basic 
set  of  performance  scores  from  the  12  hours  of  criterion  assessment, 
and  a  model  of  performance  that  would  account  for  the  covariances 
among  criterion  scores.  Then  the  correlations  between  each  predictor 
score  and  each  criterion  score  for  each  MOS  were  calculated,  and  an 
analysis  of  differential  prediction  across  criterion  dimensions  within 
MOS  (e.g.,  do  different  measures  predict  different  dimensions  of 
performance  for  given  jobs)  and  across  MOS  for  each  major  criterion 
dimension  (e.g.,  do  different  measures  predict  the  same  dimension  of 
performance  for  different  jobs)  was  carried  out. 

Results 


After  analysis,  the  item  subtests  of  the  ASVAB  plus  the  four- 
hour  battery  of  experimental  tests  were  arrayed  into  24  predictor 
scores.  They  are  listed  in  Table  1. 

The  multiple  performance  measures  were  first  aggregated  into  28- 
31  basic  criterion  scores  (depending  on  the  MOS)  by  means  of  expert 
judgment  panels  and  exploratory  factor  analyses.  A  confirmatory 
analysis  procedure  was  then  used  to  test  the  fit  of  these  basic  scores 
with  alternative  models  of  the  latent  criterion  structure.  The  best 
fitting  model  included  five  content  factors  and  two  method  factors. 

They  are  shown  as  Table  2. 

The  first  validation  analyses  generated  a  24  (predictors)  by  5 
(criteria)  matrix  of  validity  coefficients  for  each  MOS.  These 
matrices  were  examined  for  the  level  of  average  validity,  for  profiles 
of  validities  across  predictors  for  each  criterion  factor,  for  patterns 
of  validities  across  the  five  factors  within  MOS,  and  validity  patterns 
across  MOS  for  each  of  the  five  criterion  factors.  The  following 
conclusions  summarize  the  results: 

•  Each  of  the  five  criterion  factors  can  be  predicted  with 
considerable  accuracy,  but  not  by  the  same  predictors. 

•  There  is  considerable  differential  prediction  across 
criterion  factors  within  each  MOS.  This  suggests  that 
different  goals  could  be  emphasized  in 
selection/classification  (e.g.,  maximizing  technical 
performance  vs .  minimizing  discipline/motivational  problems). 

•  The  only  criterion  factor  to  show  significant  differential 
prediction  across  MOS  was  the  core  technical  performance 
factor.  For  the  other  four  performance  components,  the 
same  predictor  profile  was  found  in  each  MOS. 
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Table  1 


Coding  Speed 
Number  Operations 


Table  1  (Contd.) 


Table  1  (Contd.) 

Project  A  Test  Content  and  Predictor  Composite  Scores 


Perceptual-  Psy  c  ho  m  oto  r 


Computer  Battery  Tests  Ability  Composites 


Table  1  (Contd.) 

Project  A  Test  Content  and  Predictor  Composite  Scores 


ABLE  Scales 
Self-Esteem 
Work  Orientation 
Energy  Level 


Temperament- 
Personality  composites 


Achievement  Orientation 


Conscientiousness 

Non-Delinquency 


Dependability 
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Table  1  (Contd.) 

Project  A  Test  Content  and  Predictor  Composite  Scores 


AVOICE  Scales  Vocational  Interests 
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Table  1  (Contd.) 

Project  A  Test  Content  and  Predictor  Composite  Scores 


Job  Reward 

JOB  Scales  Perference composites 
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The  Basic  Issue 


Using  Project  A  results,  optimal  prediction  equations  can  be 
developed  for  19  MOS  and  classification  efficiency  can  be  examined 
across  the  same  19.  However,  the  Army  must  select  and  assign  people 
to  approximately  275  MOS.  When  implemented,  the  Project  B  algorithm 
must  lead  to  decision  for  all  applicants. 

There  are  three  major  ways  to  approach  this  issue: 

•  Empirical  validation  could  be  carried  out  for  all  275  MOS. 

•  Since  the  19  MOS  were  selected  to  be  representative  of 
clusters  of  MOS  judged  to  be  similar  in  content  within  each 
cluster,  validity  generalizations  could  be  assumed  within 
each  cluster  and  examined  empirically  across  the  19.  That 
is,  the  significant  differential  prediction  across  MOS  for 
the  Core  Technical  Proficiency  (CTP)  factor  may  be  accounted 
for  by  fewer  than  19  equations. 

•  A  synthetic  validation  procedure  could  be  used  to  select  a 
predictor  battery  for  each  MOS.  The  19  MOS  in  the  Project 
A  sample  provide  a  means  for  empirically  validating  any 
such  synthetic  procedures. 

It  is  the  latter  strategy  which  is  the  focus  of  the  Synthetic 
Validation  Project.  If  a  successful  synthetic  validation  procedure 
could  be  developed,  it  would  provide  a  less  costly  way  (and  perhaps 
the  only  feasible  way)  of  developing  selection/classification 
procedures  for  new  MOS,  for  MOS  that  have  undergone  significant 
changes,  or  for  MOS  that  have  relatively  few  people  in  them 

The  remaining  chapters  in  this  report  describe  the  major  parts 
in  this  synthetic  validation  effort,  a  brief  overview  of  which  is 
given  below. 


THE  SYNTHETIC  VALIDATION  PROJECT 

The  "synthetic  validity"  approach  was  first  introduced  by  Lawshe 
(1952)  as  an  alternative  to  the  situational  validity  approach,  which 
requires  separate  validity  analyses  for  each  job  in  the  organization. 
Balma  (1959)  defined  synthetic  validity  as  "discovering  validity  in  a 
specific  situation  by  analyzing  jobs  into  their  components,  and 
combining  these  validities  into  a  whole." 

Guion  (1976)  provides  a  review  of  several  approaches  to  conducting 
synthetic  validation.  The  approach  most  relevant  to  the  problem  at 
hand  involves 

•  identifying  job  components  that  are  common  across  a  range 
of  jobs 
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•  using  criterion-related  validity  procedures  to  validate 
potential  predictors  of  each  component  of  job  performance 
(pooling  incumbents  across  jobs  as  appropriate),  and 

•  developing  predictor  composites  for  each  job  by  combining 
the  prediction  equations  for  each  of  the  job  components 
that  are  relevant  to  the  job. 

The  usefulness  of  this  variant  of  synthetic  validation  depends 
on  three  critical  operations. 

First,  a  set  of  components  must  be  identified  that  cover  all 
important  aspects  of  performance  in  all  enlisted  jobs.  The  taxonomy 
of  job  components  must  be  reasonably  exhaustive  of  the  job  population 
such  that  the  critical  parts  of  any  particular  job  can  be  described 
completely  by  the  weighted  sum  of  all  relevant  components  and  very 
little  else.  In  addition,  there  must  be  a  group  of  subject  matter 
experts  (SMEs)  available  who  understand  these  components  well  enough 
to  provide  reliable  and  accurate  importance  or  relevance  weights  for 
the  components  in  a  particular  job. 

Second,  it  must  be  possible  to  establish  equations  for  predicting 
performance  on  each  component  from  current  or  potential  selection 
measures.  The  prediction  equation  for  a  given  component  must  be 
independent  of  the  particular  job  for  which  the  component  is  judged 
relevant.  Either  empirical  or  a  combination  of  empirical  and  judgment- 
based  procedures  must  be  used  to  establish  the  predictive  relationships 
for  each  component.  There  also  must  be  reliable  differences  between 
the  prediction  equations  for  different  components.  To  the  extent 
that  the  same  measures  predict  all  components  of  performance,  the 
overall  prediction  equations  will  necessarily  be  the  same  across  jobs 
(since  it  will  not  matter  which  components  are  relevant).  In  such  a 
case  a  validity  generalization  model  would  apply,  and  there  would  be 
no  basis  for  differential  classification. 

Third,  synthetic  validation  models  assume  that  overall  job 
performance  can  be  expressed  as  the  weighted  or  unweighted  sum  of 
individual  performance  components.  Composite  prediction  equations 
are  typically  expressed  as  the  corresponding  sum  of  the  individual 
component  prediction  equations.  To  estimate  the  validity  of  the 
composite  prediction  equation,  validity  estimates  for  the  predictors 
of  each  component  are  needed  and  some  further  assumptions  are  required. 
Most  typically,  it  is  assumed  that  errors  in  estimating  different 
components  of  performance  are  uncorrelated. 

Project  Design 

The  general  design  of  the  Synthetic  Validation  Project  is  as 
follows.  After  a  thorough  literature  search,  we  outlined  a  set  of 
alternative  methods  for  describing  job  components.  These  were  based 
on  our  own  and  previous  work  in  constructing  taxonomies  of  human 
performance  (e.g.,  Fleishman  and  Quaintance,  1984).  There  are  four 
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principal  kinds  of  components  or  descriptive  units  for  analyzing 
jobs:  behavior  description  approaches  (e.g.,  handling  objects), 
behavior  requirements  approaches  (e.g.,  decision  making),  ability 
requirements  approaches  (e.g.,  finger  dexterity),  and  task 
characteristics  approaches  (e.g.,  fires  main  gun). 

After  an  initial  review  of  alternative  types  of  components,  we 
decided  to  combine  behavior  requirements  and  ability  requirements  and 
to  proceed  with  three  approaches.  The  first  is  a  Job  Behaviors 
Model.  The  components  are  defined  as  general  job  behaviors  that  are 
not  task  specific,  but  which  can  underlie  several  job  tasks.  Examples 
might  be  "recalling  verbal  information"  or  "driving  heavy  equipment." 
For  this  approach  we  are  attempting  to  identify  a  set  of  behaviors  that 
can  be  linked  reliably  to  predictor  measures.  Some  concerns  are  that 
it  may  be  difficult  to  develop  the  taxonomy  of  behavior  in  sufficient 
detail  to  be  useful,  the  judgments  of  job  relevance  may  be  difficult, 
and  these  descriptions  may  not  be  accepted  by  those  making  the 
judgments. 

The  descriptive  units  in  the  second  approach  are  Job  Tasks.  An 
initial  list  of  performance  tasks  was  developed  in  Project  A  from 
duty  area  descriptions  for  the  111  enlisted  jobs  with  the  largest 
number  of  incumbents.  These  descriptions  provide  a  basis  for  defining 
job  components  that  are  clusters  of  tasks  rather  than  behaviors  within 
tasks.  The  chief  advantages  of  this  model  are  a  close  match  to 
previous  empirical  validity  data  and  the  familiarity  of  SMEs  with 
these  kinds  of  descriptions.  The  primary  concerns  are  that  the 
taxonomy  may  not  be  complete  enough  to  handle  new  jobs  and  that  the 
relationships  of  job  component  performance  to  individual  predictors 
may  be  difficult  to  determine  reliably  and  accurately. 

The  final  approach  is  an  Individual  Attribute  Model.  In  this 
approach,  the  components  are  job  requirements  described  in  terms  of 
mental  and  physical  abilities,  interests,  traits,  and  other  individual 
difference  dimensions.  This  model  eliminates  the  need  to  establish 
links  between  predictors  and  job  components  because  the  job  components 
are  the  predictors.  The  chief  concern  with  this  approach  is  that 
there  may  be  no  SMEs  who  know  enough  about  both  the  job  and  the  human 
attribute  dimensions  to  describe  job  requirements  accurately.  Also, 
this  approach  may  not  be  as  acceptable  as  a  method  based  on  more 
specific  job  descriptors. 

Procedure 


The  Synthetic  Validation  Project  follows  an  iterative  procedure. 
The  iterative  approach  provides  an  opportunity  for  revisions  of  the 
models  and  research  methods  followed  by  evaluation  of  a  more  refined 
version  of  each  approach.  The  design  specified  first  a  series  of 
exploratory  workshops  to  assess  the  completeness  and  clarity  of  each 
job  component  model  followed  by  three  phases  of  further  development 
and  evaluation.  In  Phase  I,  initial  procedures  were  tested  for  three 
of  the  Project  A  MOS.  In  Phase  II,  revised  procedures  will  be  tested 


for  seven  more  Project  A  MOS.  Final  procedures  will  be  tested  in 
Phase  III  for  nine  more  Project  A  MOS  and  four  MOS  not  sampled  by 
Project  A. 

Throughout  the  project  design,  the  emphasis  is  on  the 
identification  and  evaluation  of  alternative  approaches  to  the 
implementation  of  synthetic  validation  procedures.  We  will  evaluate 
the  extent  to  which  each  of  the  three  models  can  meet  the  assumptions 
for  synthetic  validation  and  the  extent  to  which  each  leads  to  an 
optimal  predictor  battery.  In  the  course  of  doing  that,  we  will 
compare  the  results  produced  by  different  types  of  judges  when 
evaluating  the  relevance  of  the  different  types  of  components  for  the 
target  jobs.  For  example,  are  officers  or  NCOs  the  better  judges  of 
which  task  components  are  the  most  relevant  for  a  specific  job?  Are 
psychologists  or  Army  officers  the  better  judges  of  which  attributes 
will  predict  performance  on  a  particular  job?  The  criticality 
judgments  produced  by  the  type-of-judge/type-of*component  combinations 
will  be  compared  in  terms  of  their  distributional  properties, 
interjudge  reliabilities,  discriminability,  and  acceptability. 

For  the  job  task  and  job  behavior  based  approaches,  a  particular 
job  must  first  be  described  in  terms  of  its  most  relevant  task  or 
behavior  content.  Once  the  content  elements  are  known,  the  expected 
validity  of  the  attributes  (predictor  variables)  must  then  be  estimated 
so  as  to  select  a  potential  predictor  battery.  If  a  profile  of 
attribute  validities  is  estimated  for  each  of  several  job  components, 
then  the  validity  profiles  can  be  intercorrelated  and  the  predictor 
covariances  can  also  be  estimated.  As  an  alternative,  the  validity 
of  a  measure  of  each  attribute  of  predicting  job  performance  can  be 
estimated  directly  (by  either  psychologists  or  Army  personnel)  and 
the  prior  judgment  of  which  tasks  or  behaviors  are  relevant  for  the 
job  is  eliminated. 

Again,  the  basic  objective  is  to  determine  which  combination  of 
descriptive  component,  type  of  judge,  and  type  of  judgment  leads  to 
the  best  predictor  battery.  As  used  here,  the  term  "best"  refers  to 
the  optimal  procedures  for; 

•  minimizing  the  cost  of  producing  the  synthetic  validity 
estimate  in  terms  of  time,  number  of  judgps,  number  of 
judgments,  etc. 

•  maximizing  the  discriminability  across  jobs 

•  minimizing  testing  time  during  actual  operational  use  (i.e., 
a  battery  with  fewer  tests) 

e  maximizing  the  correspondence  of  the  synthetic  prediction 

equations  with  the  empirical  ones  as  portrayed  in  the  Project 
A  data. 
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Expected  Outcomes 

Analyses  conducted  by  Project  A  have  provided  evidence  for 
reliably  differential  prediction  of  technical  proficiency  in  different 
MOS  (Wise,  Campbell,  &  Peterson,  1987).  Analyses  based  on  the  ASVAB 
subtests  alone  indicate,  for  example,  that  the  "technical"  ASVAB 
subtests  (Mechanical  Comprehension,  Auto-Shop  Information,  and 
Electronic  Information)  provide  excellent  performance  prediction  for 
many  technical  jobs  and  relatively  poor  prediction  for  most  clerical 
jobs.  New  predictors  developed  for  Project  A  similarly  show 
differential  validity  across  jobs.  The  Combat  Interest  scale 
(Peterson,  1987),  for  example,  shows  significantly  greater  validity  for 
predicting  performance  in  combat  jobs  than  in  noncombat  jobs.  These 
results  lead  us  to  expect  some  reasonable  success  in  identifying 
different  predictor  composites  for  different  jobs.  Therefore,  efforts 
to  develop  refined  procedures  for  composite  identification  appear 
justified. 

The  approach  to  selection  composite  development  recommended  as  a 
result  of  this  project  may  be  at  any  of  several  levels  of  specificity. 
At  one  extreme,  we  might  recommend  grouping  jobs  into  a  relatively 
small  number  of  families  based  on  similarities  in  job  component 
profiles.  A  different  predictor  composite  would  be  identified  for 
each  family,  but  no  attempt  at  differential  prediction  would  be  made 
^or  jobs  within  the  same  family.  The  job  description  task  would 
onsist  of  forming  a  job  components  profile  for  a  new  job  and  then 
matching  it  to  one  of  the  job  family  profiles. 

Intermediate  results  might  include  identification  of  a  much  larger 
set  of  distinct  job  families  or  procedures  for  synthesizing  composites 
based  on  a  small  number  of  general  job  components.  In  the  latter 
approach,  each  job  might  have  a  distinct  composite  if  job-specific 
weights  were  developed  for  the  small  number  of  different  prediction 
equations  corresponding  to  the  different  job  components. 

At  the  other  extreme,  we  might  demonstrate  the  usefulness  of  a 
full-blown  synthetic  validation  approach.  Each  job  would  have  a 
distinct  predictor  composite  derived  from  a  profile  that  encompassed 
a  large  number  of  detailed  job  components. 

Again,  the  overall  objective  is  to  evaluate  the  major  alternative 
synthetic  validation  procedures  as  a  means  for  producing  a 
selection/classification  predictor  battery  for  each  job  in  a  population 
of  jobs  and  for  new  jobs  that  enter  the  system  in  the  future.  The 
Army's  situation  is  such  that  a  true  classification  strategy  is 
required.  Projects  A  and  B  have  provided  both  the  validation  data 
and  an  assignment  algorithm  for  determining  optimal  decision  strategies 
and  estimating  their  limits.  It  remains  for  validity  generalization 
or  synthetic  validation  to  provide  decision  rules  for  all  jobs  in  the 
population. 
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The  remaining  chapters  in  this  report  describe  the  major  parts 
of  Phase  I  of  the  Synthetic  Validation  Project  in  more  detail. 
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SUMMARY 

Synthetic  validation  involves  disaggregating  jobs  into  components, 
identifying  valid  predictors  for  the  components,  and  aggregating  component 
validities.  One  of  the  first  steps  is  to  develop  a  job  component  model 
for  describing  jobs  in  terms  of  components.  In  the  Synthetic  Validity 
Project,  three  job  component  models  were  developed  and  tested.  This 
chapter  describes  the  development  of  the  first  two  models  and  the  evaluation 
of  the  usability  of  these  models  for  different  types  of  judges.  The  third 
model  is  described  in  the  following  chapter. 

The  Task  Category  Model  is  composed  of  major  job  tasks  that  describe 
the  content  of  jobs  in  the  population.  At  the  most  general  level,  the 
taxonomy  encompasses  five  categories:  mechanical,  general  operations, 
administrative/clerical,  combat,  and  leadership. 

The  Job  Activity  Model  is  composed  of  general  job  behaviors  that  may 
be  relevant  for  several  specific  job  tasks.  The  initial  taxonomy  was 
constructed  from  categories  suggested  by  widely-used  job  analysis 
instruments  such  as  the  Position  Analysis  Questionnaire  (PAQ)  and  elaborated 
by  research  and  theory  in  cognition,  problem  solving,  and  interpersonal 
behavior.  At  the  most  general  level,  the  taxonomy  is  composed  of 
leadership,  communication,  information  manipulation,  perceptual  judgments, 
problem  solving,  operating  equipment,  adjusting,  driving,  aiming,  and 
other  physical  actions. 

Army  subject  matter  experts  used  instruments  based  on  these  job 
component  models  to  describe  jobs.  Results  showed  that  there  was  adequate 
agreement  (single-rater  reliability)  in  their  judgments  of  relevance  of  a 
job  component  for  a  job.  The  Task  Category  instrument  yielded  more  reliable 
judgments  than  the  Job  Activity  instrument.  Officers  were  somewhat  more 
consistent  than  NCOs  on  the  Task  instrument  and  Field  judges  were  more 
consistent  than  School  judges  on  the  Activity  instrument.  In  addition, 
there  was  also  adequate  discriminability  in  their  job  descriptions  for 
different  jobs.  Finally,  judges  estimated  that  each  taxonomy  covered 
about  70  percent  of  the  job  content. 
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CHAPTER  3: 


Analysis  of  Job  Components: 

The  Development  and  Evaluation  of  Alternative  Methods 


As  indicated  in  chapter  1,  the  Synthetic  Validation  Project 
incorporates  an  iterative  approach  for  developing  and  testing  multiple 
models  for  synthetic  validation.  One  of  the  first  steps  in  synthetic 
validation  is  the  decomposition  of  jobs  into  job  components. 

Nature  of  Job  Component  Models 

Four  different  bases  for  defining  job  component  models,  or  decompos¬ 
ing  jobs  into  components,  were  reviewed  as  a  first  step  in  model  develop¬ 
ment.  Three  of  these  were  judged  reasonable  for  the  Army's  purposes  of 
describing  jobs  and  linking  components  with  selection  measures.  This 
chapter  will  focus  on  two  of  these  three  job  component  models,  describing 
their  descriptor  bases,  linkages,  and  required  judgments  and  potential  or 
preferred  judge  types.  The  third  model,  called  the  Attribute  Model,  is  an 
attempt  to  describe  jobs  directly  in  terms  of  the  required  skills  and 
other  attributes  measured  by  the  selection  battery.  The  development  and 
evaluation  of  the  predictor  taxonomy  that  is  the  basis  of  this  model  will 
be  described  in  the  next  chapter. 

One  model  developed  for  the  Synthetic  Validation  Project  is  based  on 
what  is  commonly  called  the  "Task  Characteristics"  approach,  which  has  as 
its  basis  the  work  itself,  or  the  inherent  characteristics  of  the  tasks 
themselves.  The  performer's  observable/overt  behavior  and  internal 
processing  are  not  taken  into  account.  The  task-based  model  came  to  be 
referred  to  as  the  "Task  Category"  model . 

The  second  model  was  developed  on  the  basis  of  descriptors  referred 
to  as  "behaviors"  in  the  literature,  and  contains  components  found  in 
"Behavior  Description"  models.  Observations  and  descriptions  of  worker 
behaviors  while  working  are  the  units  of  analysis.  The  components  of 
these  models  are  thus  based  on  observable  behaviors,  not  on  behaviors  that 
are  expected  or  required.  The  model  developed  for  this  project  is  called 
the  "Job  Activity"  model,  composed  of  "activities." 

The  choice  of  job  component  model  used  for  synthetic  validation 
purposes  influences  decisions  concerning  how  to  go  about  establishing  the 
linkages  for  synthetic  validation.  These  decisions  basically  answer  the 
questions: 

•  what  types  of  linkages  are  required? 

•  what  types  of  judgments  are  required? 

f  who  should  the  judges  be? 

Both  the  task-based  and  activity-based  models  require  participation 
of  expert  judges  to  support  the  linkages.  We  will  briefly  summarize  some 
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findings  from  the  literature  that  have  implications  for  the  judgment 
procedures  which  accompany  use  of  these  models. 

Types  of  Linkages 

Essentially,  the  job  component  model  in  a  synthetic  validation 
application  serves  as  a  mechanism  for  relating  a  predictor  domain  to  a  job 
performance  domain.  Thus,  there  are  two  types  of  linkages: 

•  predictor- job  component  linkages 

§  job  component-overall  job  linkages 

Applications  have  demonstrated  that  these  linkages  can  be  established 
empirically  and/or  judgmental ly. 

Type  of  Judgment 

We  need  to  specify  just  what  types  of  judgments  are  reouired  to 
establish  the  linkages  of  the  model.  The  job  description  judgments  are 
those  typically  collected  with  structured  checklists  or  instruments  to 
assess  degree  of  criticality  or  importance  of  specific  components  for  job 
performance.  These  ratings  may  be: 

•  part  vs.  not  part  of  job 

•  importance 

•  frequency 

•  percent  of  time  spent 

t  difficulty,  etc. 

On  the  other  hand,  predictor-job  component  linkages  require  that  judges 
estimate  the  degree  to  which  relationships  exist  between  selection  measures 
and  job  components.  These  may  be  relative  validity  estimates  (e.g.,  using 
a  rating  scale)  or  estimates  of  correlation  coefficients. 

Judges 

Two  main  questions  concerning  judges  are  typically  addressed  in  the 
literature: 

•  who  should  the  judges  be--what  rater  group  is  appropriate  for 
providing  judgments  required  at  several  points  in  the  synthetic 
validation  process 

•  what  qualifications  should  the  specific  raters  have,  in  order  to 
provide  reliable,  accurate  information? 

First,  judges  will  provide  job  descriptive  information  in  the  form  of 
estimates  of  criticality  of  job  components  for  job  performance.  In 
addition,  validity  estimates  will  be  needed  to  link  predictor  measures  to 
job  components.  These  two  types  of  judgments  are  very  different;  thus, 
judges  with  different  areas  of  expertise  should  provide  them.  Optimally, 
Army  subject  matter  experts  (SMEs)  should  provide  the  job  descriptive 
judgments,  while  psychologists  knowledgeable  in  the  areas  of  test 
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development,  test  validation,  measurement,  and  individual  differences 
should  provide  judgments  to  link  predictors  with  job  components. 


Based  on  our  review  of  the  rater  literature,  we  can  draw  some 
conclusions  and  offer  some  suggestions  for  designating  judges  for  providing 
job  description  and  validity  judgments: 

t  reliable  and  accurate  ratings  can  be  obtained  from  incumbents, 
supervisors,  and  job  analysts 

e  potential  gains  in  rating  quality  could  result  from  selecting 
judges  who  have  proven  to  be  "effective"  performers  in  their  own 
jobs 

•  empirical  validities  can  be  closely  approximated  by  judged 
validities  for  a  variety  of  tests,  criteria,  and  jobs 

•  more  experienced  experts  provide  better  validity  estimates. 

We  have  explained  the  synthetic  validation  model  to  guide  the 
development  of  a  job  taxonomy  and  method  for  entry-level  MOS.  We  have 
also  examined  the  issues  regarding  synthetic  validation,  job  taxonomies, 
and  expert  judgments.  Now  we  will  describe  the  development  and  results  of 
field  testing  of  the  Task  Category  and  Job  Activity  models. 

Development  of  Job  Component  Models 


Task  Category  Model 

Initial  development.  In  developing  the  task  category  taxonomy, 
project  staff  relied  on  prior  job  analysis  information  and  validation 
data.  Two  sources  of  job  analysis  information  guided  the  development  of 
the  initial  task  taxonomy.  First,  project  staff  reviewed  general  job 
descriptions  (contained  in  Army  Regulation  611-201)  to  determine  job  tasks 
that  were  performed  by  a  soldier  across  a  number  of  jobs.  A  representative 
sample  of  111  jobs  were  selected.  The  sample  of  jobs  had  been  previously 
clustered  into  23  job  clusters  by  personnel  experts  familiar  with  Army 
jobs.  Initial  task  performance  categories  were  derived  from  general  job 
descriptions  in  each  job  cluster.  The  second  source  for  identifying 
common  job  tasks  was  critical  incidents  collected  to  develop  performance 
rating  scales.  In  deriving  common  job  tasks,  emphasis  was  placed  on 
determining  what  a  soldier  in  each  job  might  be  observed  doing  and  what  he 
or  she  might  be  trying  to  accomplish.  Therefore,  each  resulting  category 
was  constructed  to  represent  a  set  of  actions  leading  to  an  objective. 

From  the  job  descriptions  and  the  critical  incidents,  53  categories  were 
initially  identified.  These  53  task  categories  were  used  as  criterion 
categories  for  gathering  predictor-criterion  validity  judgments  for  Project 
A  (Wing,  Peterson,  i  Hoffman,  1984). 

Development  of  the  task  performance  taxonomy  was  also  guided  by 
results  of  empirical  validation.  Two  major  findings  from  the  Project  A 
concurrent  validation  influenced  the  emerging  task  taxonomy.  First,  the 
validation  results  suggested  a  basic  division  of  job  performance  measures 
into  technical  proficiency  ("can-do")  and  nontechnical  aspects  of 
performance  ("will  do").  Second,  results  of  the  predictor-criterion 
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relationships  revealed  three  or  four  patterns  of  predictor  relationships 
for  jobs  as  a  whole.  Mechanical  jobs  and  administrative  jobs  had  the 
greatest  differences  in  predictor  relationships.  Combat  and  other  jobs 
had  intermediate  patterns  of  predictor  relationships.  Interest  and  other 
noncognitive  predictors  added  to  the  discrimination  of  combat  families 
from  other  families  of  jobs. 

The  preliminary  performance  taxonomy  that  emerged  from  these  sources 
was  subsequently  adapted  to  cover  a  wider  range  of  jobs  in  terms  of 
performance  elements.  As  part  of  Project  A's  planning  for  further  MOS 
family  development,  we  revised  and  expanded  the  original  53  task  performance 
categories.  SpeciHcally,  we  examined  all  of  the  Army  MOS  definitions 
from  AR  611-201  that  were  not  originally  reviewed  and  added  categories  to 
cover  the  new  job  duties.  We  also  added  variables  to  insure  coverage  of 
the  criterion  domain  as  defined  by  Project  A's  MOS  training  duties  areas 
(identified  from  the  Army  Occupational  Survey  Program  results),  behavioral ly 
anchored  rating  scale  (BARS)  dimensions,  and  its  MOS  task  clusters.  Then 
the  revised  list  was  reviewed  by  a  group  of  Army  officers  during  one  of 
Project  A's  utility  workshops.  Based  on  these  additional  reviews  and 
further  external  and  internal  suggestions,  the  list  of  task  performance 
categories  was  expanded  to  88  task  categories.  In  addition  to  our  attention 
to  coverage,  we  also  tried  to  use  language  that  would  be  understood  by 
prospective  Army  SMEs. 

Task  categories  taxonomy  and  subsequent  revisions.  The  initial 
taxonomy  is  shown  in  Figure  1.  Technical  proficiency  was  initially  divided 
into  four  domains:  maintenance,  combat,  general,  and  administration. 
Nontechnical  proficiency  was  composed  of  effort  and  leadership,  personal 
discipline,  and  physical  fitness  and  military  bearing.  Examples  of  detailed 
descriptions  of  several  task  category  components  are  provided  in  Figure  2. 
The  Task  Category  taxonomy  used  in  Phase  I  is  shown  in  Appendix  A. 

A  number  of  revisions  were  made  to  the  individual  categories  and  the 
taxonomy  after  initial  testing  with  Army  SMEs.  Some  of  the  changes  were: 

•  narrowing  down  the  frame  of  reference  for  describing  jobs  to 
focus  on  the  job  a.s  performed  by  soldiers  who  have  completed 
basic  training  and  advanced  individual  training  and  have  18 
months  on-the-job  experience, 

•  clarifying  the  wording  of  items  which  resulted  in  large  standard 
deviations  of  2.0  or  more  in  the  prepilot  test  relevance  ratings, 

•  collecting  a  frequency  rating  for  the  overall  job  followed  by 
importance  ratings  for  core  technical  proficiency,  general 
soldiering  proficiency,  and  the  overall  job, 

•  adding  more  job  components  to  the  model. 


,  Pros  and  cons  of  model .  Based  on  Project  A  results  (i.e.,  content 
validation  data),  much  is  known  about  the  taxonomy  and  its  components.  As 
a  great  deal  of  military  training  and  testing  is  based  on  task  performance, 
SMEs  have  considerable  experience  with  judgments  on  the  components.  The 
method  yields  an  easy  match  between  job  analysis  information  and  Project  A 
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validity  data  because  the  Task  Categories  taxonomy  and  criterion  measures 
were  founded  on  the  same  theoretical  framework  of  job  performance.  On  the 
other  hand,  there  are  several  shortcomings  associated  with  using  a  task- 
based  model.  As  the  taxonomy  was  derived  from  existing  jobs,  it  may  not 
be  comprehensive  enough  for  describing  new  jobs.  In  addition,  the  method 
requires  a  large  num’jer  of  judgments  from  SMEs  because  of  the  large 
number  of  task  categories.  In  Phase  I  for  example,  the  Task  Categories 
Taxonomy  contained  96  elements  compared  to  53  elements  for  the  Job  Activity 
Taxonomy. 


I.  TECHNICAL  COMPONENTS 

A.  Maintenance  Components 
Troubleshoot 
Inspect 
Repair 

Install/Operate 

6.  Combat  Components 

Use  Individual  Weapons/Combat 
Operate  Weapon  Systems 
Identify  Targets 

Navigation/Determine  Distance  and  Direction 
Safety/Survival 

C.  General  Components 

Equipment/Vehicle  Operation 
Construction  and  Structural  Repair 
Pack/Load  Materials 
Technical  Procedures 


D.  Administrative  Components 
Clerical 

Record  Keeping/Scheduling 
Communication/Reporting 
Analyze  Data 
Graphics/Drafting 


II.  NONTECHNICAL  COMPONENTS 

A.  Effort  and  Peer  Leadership 

B.  Personal  Discipline 

C.  Physical  Fitness  and  Military  Bearing 


Figure  1.  Initial  Task  Category  Taxonomy 
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I.  TECHNICAL  COMPONENTS 

Ai  Maintenance  Components 

Troubleshoot 

15.  Troubleshoot  electronic  components  --  find  the  cause  of  malfunctions 
in  electronic  equipment  and  components  using  technical  manuals, 
tools,  and  test  equipment  (e.g.  specialized  test  sets). 

12.  Troubleshoot  electrical  systems  --  find  the  cause  of  malfunctions  in 
electrical  parts  and  equipment  using  technical  manuals,  tools,  and 
test  equipment  (e.g.  multimeters,  test  lights). 

6.  Troubleshoot  mechanical  systems  --  find  the  cause  of  malfunctions  in 
mechanical  parts  and  equipment  using  technical  manuals,  tools,  and 
test  equipment  (e.g.  calipers,  gauges,  torque  wrenches). 

9.  Troubleshoot  hydraulic  or  pneumatic  systems  --  find  the  cause  of 
malfunctions  in  hydraulic  or  pneumatic  parts  and  equipment  using 
technical  manuals,  tools,  and  test  equipment  (e.g.  pressure  gauges). 

2.  Troubleshoot  weapons  --  find  the  cause  of  malfunctions  in  weapons 
using  technical  manuals,  tools,  and  test  equipment. 

Inspect 

14.  Inspect  electronic  systems  --  measures  use  test  equipment  and  manuals, 
and  observe  electronic  systems  (e.g.  communications  equipment, 
radar,  missile  and  tank  computer  ballistics)  to  detect  problems  and 
malfunctions. 

II.  Inspect  electrical  systems  --  measure,  use  test  equipment  and  manuals, 
and  observe  electrical  systems  (e.g.  generators,  wiring  harnesses, 
switches,  relays,  circuit  breakers)  to  detect  problems  and 
malfunctions. 

5.  Inspect  mechanical  systems  --  measure,  use  test  equipment  and  manuals, 
and  observe  mechanical  equipment  (e.g.  engines,  transmissions(  to 
detect  problems  and  malfunctions. 

8.  Inspect  hydraulic  or  pneumatic  systems  --  measure,  use  test  equipment 
and  manuals  and  observe  hydraulic  or  pneumatic  parts  and  equipment 
(e.g.  brakes,  hydraulics,  refrigeration  systems)  to  detect  problems 
and  malfunctions. 


Figure  2.  Examples  of  detailed  task  categories 
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Job  Activity  Model 


Initial  development.  The  descriptor,  or  unit  of  analysis,  for  this 
approach  is  termed  the  general  job  behavior  and  is  defined  as  a  behavioral 
unit  that  is  not  specific  to  particular  tasks  but  can  be  an  important 
component  of  several  job  tasks.  For  example,  typing  a  letter  on  a 
typewriter  and  typing  a  letter  on  a  Personal  Computer  are  two  job  tasks. 
Operating  a  keyboard  is  a  general  job  behavior  that  is  common  to  each 
task.  To  develop  a  relevant  taxonomy  of  behaviors  with  which  to  describe 
the  behavioral  requirements  of  entry  level  Army  jobs  (MOS),  we  first 
consulted  major  published  instruments  such  as  the  Position  Analyses 
Questionnaire  (McCormick,  Jeaneret,&  Mecham,  1972),  Occupation  Analyses 
Inventory  (Cunningham,  Boese,  Neeb,  &  Pass,  1983),  and  Functional  Job 
Analyses  (Fine,  1962,  1963,  1974)  scales.  A  basic  review  of  these 
instruments  is  presented  in  Fleishman  and  Quaintance  (1984).  Although 
none  of  the  instruments  is  devoted  exclusively  to  items  that  reflect 
behavioral  units,  as  defined  above,  each  of  them  provided  ideas  about  the 
domains  of  behaviors  that  should  be  included  in  the  initial  taxonomy. 

For  additional  ideas,  we  also  examined  as  many  relevant  unpublished 
checklists  as  we  could  find  through  personal  contacts. 

An  initial  list  of  potentially  relevant  items  was  compiled  from 
these  sources  and  submitted  to  each  member  of  the  projecL  staff  for 
comment,  review,  and  suggested  revisions.  Much  discussion  ensued  as  to 
the  most  useful  definition  for  a  job  behavior,  the  boundaries  of  the 
total  domain,  and  the  level  of  specificity  at  which  the  items  should  be 
written. 

The  next  step  was  to  specify  a  possible  structure,  or  set  of  higher 
order  categories  for  the  taxonomy.  We  were  all  reminded  at  thisw^int 
that  psychology  as  a  discipline  has  never  produced  an  overall  behavioral 
taxonomy.  There  is  much  research  in  specific  areas  such  as  problem 
solving/decision  making,  participation  in  groups,  reading,  etc.  but  no 
one  has  ever  tried  (perhaps  with  good  reason)  to  develop  one  overall 
taxonomy  of  human  behavior.  Similarly,  no  latent  structure  of  job  behavior 
has  been  developed.  However,  for  suggestions  as  to  what  a  higher  order 
structure  for  the  taxonomy  might  include,  we  reviewed  topics  included  in 
the  Annual  Review  of  Psychology  for  the  past  ten  years  for  possible 
relevance  to  Army  MOS.  For  example,  various  theories  of  leadership 
suggest  potential  categories  of  leadership  behavior  (e.g.,  Yukl,  1981). 
Fleishman  and  Quaintance  (1984)  also  review  several  specialized  behavior 
categorization  schemes  (e.g.,  Berliner,  Angell,  &  Shearer,  1964). 

After  several  iterations  of  editing  and  revisions  of  the  input 
general  by  the  above  steps,  an  initial  taxonomy  hierarchical  in  nature, 
was  generated  and  submitted  to  pilot  testing  with  Army  SMEs. 

Job  activities  taxonomy  and  subsequent  revisions.  The  initial 
taxonomy  consisted  of  job  behaviors  at  3  levels  of  generality.  At  the 
most  general  level,  there  were  seven  categories. 
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•  interpersonal  behaviors 

•  spealcing  behaviors 

•  writing  behaviors 

•  cognitive  behaviors 

•  complex  problem  solving 

•  operating  equipment 

t  physical  activities 

A  detailed  presentation  of  the  initial  taxonomy  can  be  found  in  Appendix  B. 
The  current  job  activity  taxonomy  which  consists  of  53  components  is  found 
in  Appendix  D.  The  initial  and  current  taxonomy  in  Appendices  B  and  C  show 
the  three  levels  of  generality  of  the  components. 

Subsequent  revisions  of  the  job  activity  taxonomy  and  its  components 
paralleled  those  made  to  the  Task  Category  taxonomy.  Again,  the  revisions 
were: 

•  narrowing  down  the  frame  of  reference  for  describing  jobs  to 
focus  on  the  job  as  performed  by  soldiers  who  have  completed 
basic  training  and  advanced  individual  training  and  have  18 
months  on-the-job  experience, 

•  clarifying  the  wording  of  items  which  resulted  in  large  standard 
deviations  of  2.0  or  more  in  the  prepilot  test  relevance  ratings, 

•  collecting  a  frequency  rating  for  the  overall  job  followed  by 
importance  ratines  for  core  technical  proficiency,  general 
soldiering  proficiency,  and  the  overall  job, 

•  adding  more  job  components  to  the  model. 


Pros  and  cons  of  model .  The  major  strength  of  this  approach  is  that 
in  comparison  to  the  Task  Category  components,  job  description  using  job 
activities  allows  for  easier  mapping  to  the  predictor  tests,  most  of  which 
are  also  behaviorally  based.  There  are  a  number  of  disadvantages  to  using 
a  job  activities  model  for  synthetic  validation.  First,  as  the  categories 
are  composed  of  general  job  behaviors,  it  may  be  difficult  to  complete  the 
taxonomy  in  sufficient  detail  to  be  useful.  Second,  a  difficult  judgment 
is  needed  to  link  validity  information  to  the  job  component  information. 

The  difficulty  is  in  linking  the  broad  job  behaviors  (which  may  span 
several  tasks)  with  the  criterion  measures.  Third,  a  Job  Activity  model 
may  have  less  face  validity  and  probably  less  acceptability  among  Army 
SMEs. 

A  job  component  model  provides  the  basis  for  analyzing  jobs  into  a 
common  set  of  job  components.  The  two  job  component  models  developed  for 
analyzing  jobs  into  components  can  be  evaluated  on  the  basis  of 
reliability,  validity,  and  comprehensiveness.  Thus  we  developed  and  tested 
each  model  to  be  sure  it  produced: 

t  reliable  job  descriptions--high  interrater  agreement  in  endorsing 
importance  or  relevance  of  components  for  jobs; 
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•  different  descriptions  for  jobs  with  distinct  duties  or 
requirements; 

•  comprehensive  coverage  of  the  job  performance  domain  for  a  host 
of  jobs. 


In  addition,  we  also  investigated  whether  judges  of  different  ranks  or 
job  assignments  would  produce  judgments  of  differing  reliabilities. 


Method 


Procedure 


Two  rounds  of  workshops  were  conducted  to  test  the  instruments  based 
on  the  job  component  models.  A  Pilot  Test  was  conducted  in  December  1987 
and  January  1988;  a  Phase  I  Test  was  conducted  in  March  and  April  1988. 

At  each  round  of  data  collection,  we  requested  the  participation  of 
officers,  senior  NCOs,  and  civilians  employed  by  the  Army  to  serve  as 
SMEs  in  the  job  description  workshops  for  three  military  occupational 
specialties  (MOS).  The  three  Phase  I  MOS  were  Infantryman  (IIB),  Light¬ 
wheeled  Vehicle  Mechanic  (63B),  and  Administrative  Specialist  (711). 

Five  different  sites  were  visited:  Aberdeen  Proving  Ground  in  Maryland, 
Fort  Benning  and  Fort  Stewart  in  Georgia,  Fort  Benjamin  Harrison  in 
Indiana,  and  Fort  Ord  in  California. 

At  each  workshop,  the  workshop  administrator  first  provided  an 
overview  of  the  project  goals  and  the  purpose  of  the  workshops.  In  the 
Pilot  Test,  participants  completed  the  job  description  questionnaires. 
More  recently,  in  the  Phase  I  Test,  the  participants  completed  both  the 
job  description  questionnaires  and  the  standard  setting  questionnaires. 
The  order  of  the  questionnaires  was  counterbalanced  across  sites,  rank 
(officer,  NCO,  or  civilian),  and  job  assignment  (FORSCOM  or  DOTD). 

In  the  Pilot  Test,  participants  provided  importance,  frequency,  and 
difficulty  ratings  for  each  job  component  in  relation  to  overall  job 
performance;  whereas,  in  the  Phase  I  Test,  participants  provided 
frequency  ratings,  and  importance  ratings  for  Core  Technical  Proficiency 
(CTP),  General  Soldiering  Proficiency  (GSP),  and  Overall  Job  Performance 
(OJP). 


Efilsdii 

Extensive  analyses  were  conducted  on  the  Phase  I  data,  including: 
descriptive  statistics,  reliability  estimates,  variance  component, 
fidelity  coefficients,  and  profile  correlations.  A  complete  description 
of  these  results  is  beyond  the  scope  of  the  current  chapter,  however, 
interested  readers  can  find  more  detailed  Information  in  Appendix  E. 

RgUti,9n$h1p,  gf-RitlDgS 

Mean  within-rater  correlations  of  frequency,  importance,  and 
difficulty  ratings  are  presented  in  Table  1.  These  correlations  were 
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based  on  only  those  components  that  had  a  non-zero  frequency  rating, 
since  importance  and  difficulty  were  not  rated  for  components  that  were 
not  performed.  .  For  Task  Categories,  the  mean  within-rater  correlations 
between  importance  and  frequency  ratings  were  r«.48  for  IIB,  r».57  for 
63B,  and  r».63  for  71L.  The  mean  within-rater  correlations  between 
difficulty  and  importance  or  frequency  ratings  ranged  from  r»-.09  to 
r=.25. 


Table  1 

Pilot  Test:  Mean  Within-rater  Correlations  between  Ratings 


Ratings 

Task  Cateoorv 

IIB  63B  71L 

Job  Activity 

IIB  63B  71L 

Frequency- Importance 

.48 

.57  .63 

.56 

.70 

.70 

Difficulty-Importance 

.11 

.25  .22 

.08 

.21 

.20 

Frequency-Difficulty 

-.09 

.13  .14 

-.02 

.14 

.07 

A  similar  pattern  of  intercorrelation  was  obtained  for  the  three 
judgments  for  Job  Activities.  The  mean  within-rater  correlations  between 
importance  and  frequency  ratings  were  r».56  for  IIB,  r“.70  for  63B,  and 
r=,70  for  71L.  The  mean  within-rater  correlations  between  difficulty 
and  importance  or  frequency  ratings  ranged  from  r=-.02  to  r».21. 

These  results  suggest  that  on  both  instruments,  there  was  a  great 
deal  of  redundancy  in  importance  and  frequency  information,  but  the 
difficulty  ratings  yielded  rather  different  information. 

One  of  the  necessary  linkages  in  synthetic  validation  was  between 
the  job  components  and  job  performance  measures.  Therefore  in  the  Phase 
I  data  collection,  the  project  staff  revised  the  relevance  ratings  to 
include  importance  ratings  for  CTP,  6SP,  and  OOP.  For  Phase  I,  we 
computed  correlations  of  the  mean  rating  profiles  across  the  three  MOS 
and  these  correlations  are  displayed  in  Table  2.  For  IIB,  there  were  no 
differences  among  the  three  importance  ratings  for  CTP,  6SP,  and  OJP. 

As  expected,  rating  profiles  for  6SP  were  highly  correlated  across  the 
MOS  (jr-.BA-.BB  for  Tasks;  e».64-.85  for  Activities).  Rating  profiles  on 
CTP  should  differentiate  the  three  jobs,  and  they  did.  Only  low  to 
moderate  correlations  resulted  across  jobs.  The  Job  Activities  ratings 
provided  a  high  degree  of  discrimination  between  jobs  with  no  correlation 


38 


at  all  in  the  profiles  for  CTP.  The  Task  Category  ratings  showed 
significantly  less  discrimination  across  jobs. 

Table  2 

Correlations  of  Mean  Importance  Ratings  among  IIB.  63B.  and  7iL 
Task  Category  Instrument 


OJPllB 

0JP63B 

Rating  and 

0JP71L  CTPllB 

MOS 

CTP63B 

CTP71L 

GSPllB 

GSP63B 

0JP63B 

0.763 

0JP71L 

0.614 

0.643 

CTPllB 

0.998 

0.746 

0.596 

CTP63B 

0.544 

0.927 

0.524 

0.523 

CTP71L 

0.217 

0.356 

0.857 

0.197 

0.387 

GSPllB 

0.999 

0.767 

0.632 

0.997 

0.545 

0.237 

GSP638 

0.874 

0.968 

0.712 

0.860 

0.829 

0.366 

0.879 

GSP71L 

0.825 

0.776 

0.918 

0.812 

0.587 

0.602 

0.839 

0.873 

Job  Activity  Instrument 


OJPllB 

0JP63B 

Rating  and  MOS 

0JP71L  CTPllB  CTP63B 

CTP71L 

GSPllB 

GSP63B 

0JP63B 

0.575 

OJP71L 

0.461 

0.118 

CTPllB 

0.994 

0.574 

0.408 

CTP63B 

0.272 

0.917 

-0.034 

0.264 

CTP71L 

0.147 

-0.063 

0.916 

0.079 

-0.072 

GSPllB 

0.997 

0.581 

0.494 

0.988 

0.280 

0.187 

GSP63B 

0.847 

0.885 

0.365 

0.841 

0.654 

0.084 

0.854 

GSP71L 

0.710 

0.360 

0.919 

0.672 

0.125 

0.721 

0.736 

0.645 

•n 
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Interrater  Reliability 

Variance  component  estimates  provide  a  concise  way  of  summarizing 
differences  among  job  descriptor  means  and  describing  the  relative 
contribution  of  various  conditions  to  variance  in  the  ratings. 
Independent  variables  and  the  number  of  levels  within  each  variable  are 
as  follows: 


Independent  Variables  Number  of  Levels 

Component  96  for  Task  Category;  53  for  Job  Activity 


MOS 


3  (IIB,  63B,  and  71L) 


Rank  of  Rater  2  (NCO,  Officer;  civilians  omitted 

because  of  limited  number  of 
observations) 

Assignment  of  Rater  2  (FORSCOM,  DOTD) 

(Command) 


Rater 


160  for  Task  Category;  162  for  Job 
Activity 


As  rater  effects  were  nested  within  MOS,  rank,  and  command  (job 
assignment),  we  ran  two  separate  analyses  of  variance  models.  The 
first  model  included  only  rater  and  component.  Estimates  of  the 
variance  among  all  raters  were  obtained  from  this  model.  The  second 
model  included  component,  MOS,  rank,  and  command,  but  not  rater. 
Variance  attributable  to  terms  with  MOS,  rank,  and/or  command  were 
subtracted  from  the  overall  rater  variance  estimates  from  the  first 
model.  The  differences  provided  estimates  of  the  variance  due  to  rater 
nested  within  MOS,  rank,  and  command. 


Variance  due  to  component,  MOS,  and  the  component  by  MOS 
interaction  were  reliable  sources  of  variance  and  when  divided  by  total 
variance  in  the  ratings,  provided  estimates  of  the  average  single-rater 
reliability.  Single-rater  reliabilities  for  the  three  importance 
ratings  from  Phase  I  are  presented  in  Table  3.  For  the  Task  Category 
instrument,  reliabilities  for  CTP,  6SP,  and  OJP  were  .55,  .46,  and  .52, 
respectively.  Corresponding  values  for  the  Job  Activity  were  .43,  .30, 
and  .37.  Reliabilities  for  the  judgments  on  the  Task  instrument  were 
higher  than  those  on  the  Job  Activity  instrument. 

Some  differences  between  rater  types  were  also  found.  Table  4 
shows  the  single-rater  reliabilities  for  OJP  for  raters  of  different 
ranks  and  job  assignments.  For  the  Task  Category  instrument,  judges 
from  the  field  gave  more  consistent  ratings  than  school  judges.  For 
the  Job  Activity  instrument,  officers  were  more  consistent  than  NCOs. 

Single-rater  reliabilities  from  the  Pilot  Test  were  also  included 
in  Table  3.  The  reliability  of  the  OJP  ratings  actually  dropped  for 
Phase  I  compared  to  the  Pilot  Test.  This  may  be  due  to  changes  in  the 
rating  task.  Obtaining  separate  CTP  and'GSP  ratings  prior  to  the  OJP 
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rating  might  have  helped  judges  focus  more  on  GSP  than  before,  thus 
decreasing  the  variance  between  jobs. 

Table  3 

Summary  of  Sinale-rater  Reliability  Estimates 
for  Task  Category  and  Job  Activity  Instruments 


Task  Category  Job  Activity 


CTP 

GSP 

OOP 

Pilot 

CTP 

GSP 

OOP 

Pilot 

Overall 

0.55 

0.46 

0.52 

0.58 

0.43 

0.30 

0.37 

0.45 

Within  Rank 

0.60 

0.48 

0.54 

0.62 

0.51 

0.32 

0.41 

0.48 

Within  Command 

0.72 

0.52 

0.59 

0.63 

0.50 

0.32 

0.41 

0.47 

Within  Rank  &  Command 

0.74 

0.53 

0.60 

0.65 

0.54 

0.32 

0.43 

0.50 
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Table  4 


Comparison  of  Single-rater  Reliabilities  for  Importance  for 
Overall  Job  Performance  between  Phase  I  and  Pilot  Test 


Task  Category  Job  Activity 

Phase  I  Pilot  Phase  I  Pilot 


NCO 


Overall 

0.53 

0.61 

0.35 

0.49 

Within  Assignment 

0.54 

0.63 

0.35 

0.49 

Officer 

Overall 

0.59 

0.63 

0.47 

0.58 

Within  Assignment 

0.61 

0.64 

0.48 

0.59 

FORSCOM 

Overall 

0.60 

0.65 

0.37 

0.52 

Within  Rank 

0.61 

0.66 

0.39 

0.53 

DOTD 

Overall 

0.51 

0.63 

0.38 

0.47 

Within  Rank 

0.53 

0.64 

0.39 

0.48 

Discriminabil itv 

Summary  of  the  variance  components  results  are  presented  in 
Tables  5a  to  5c.  A  large  Component  by  MOS  interaction  suggested  that 
the  weights  were  applied  differentially  for  the  job  components  across 
MOS.  On  both  instruments,  the  variance  components  due  to  the  CTP  by 
MOS  interaction  were  larger  than  the  OJP  by  MOS  or  GSP  by  MOS 
interactions.  The  size  of  the  GSP  by  MOS  interaction  suggested  that 
General  Soldiering  tasks  played  a  greater  role  in  the  overall  rating 
during  Phase  I  than  during  the  Pilot  Test.  Therefore  a  better 
reliability  comparison  is  the  Phase  I  Core  Technical  ratings  with  the 
Pilot  Test  Overall  ratings. 
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Table  5a 

Variance  Components  for  Ratings  of  Importance  for 
Overall  Job  Performance 


Task  Activity 


Components  Variance  Percent  Variance  Percent 


Component 

1.190 

35.63% 

0.430 

14.48% 

MOS 

0.080 

2.41% 

0.087 

2.93% 

Component  x  MOS 

0.468 

14.00% 

0.579 

19.51% 

Rank 

0.021 

0.63% 

0.046 

1.56% 

Command 

0.019 

0.57% 

0.022 

0.75% 

Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Command  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  Command  x  MOS 

0.091 

2.73% 

0.173 

5.82% 

Component  x  Rank 

0.010 

0.29% 

0.067 

2.25% 

Component  x  Command 

0.000 

0.00% 

0.022 

0.74% 

Comp  X  Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.019 

0.57% 

0.000 

0.00% 

Comp  X  Command  x  MOS 

0.292 

8.75% 

0.075 

2.53% 

Cp  X  Rk  X  Cd  X  M 

0.000 

0.00% 

0.004 

0.13% 

Rater(Rk  Cd  M) 

0.151 

4.52% 

0.137 

4.62% 

Component  x  Rater 

0.999 

29.92% 

1.327 

44.67% 

Total 

3.341 

100.00% 

2.970 

100.00% 
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Table  5b 

Variance  Components  for  Ratings  of  Importance  for 
Core  Technical  Proficiency 


Task  Activity 


Components  Variance  Percent  Variance  Percent 


Component 

0.562 

18.40% 

0.113 

3.50% 

MOS 

0.230 

7.53% 

0.238 

7.38% 

Component  x  MOS 

0.897 

29.36% 

1.040 

32.24% 

Rank 

0.029 

0.95% 

0.082 

2.54% 

Command 

0.018 

0.59% 

0.076 

2.36% 

Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Command  x  MOS 

0.045 

1.47% 

0.032 

1.00% 

Rank  x  Command  x  MOS 

0.155 

5.07% 

0.328 

10.17% 

Component  x  Rank 

0.047 

1.54% 

0.096 

2.98% 

Component  x  Command 

0.005 

0.16% 

0.000 

0.00% 

Comp  X  Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.000 

0.00% 

0.011 

0.34% 

Comp  X  Command  x  MOS 

0.474 

15.52% 

0.032 

0.99% 

Cp  X  Rk  X  Cd  X  M 

0.000 

0.00% 

0.000 

0.00% 

Rater(Rk  Cd  M) 

0.030 

0.98% 

0.000 

0.00% 

Component  x  Rater 

0.563 

18.43% 

1.178 

36.51% 

Total 

3.055 

100.00% 

3.226 

100.00% 
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Table  5c 

Variance  Components  for  Ratings  of  Importance  for 
General  Soldiering  Proficiency 


Task  Activity 


Components  Variance  Percent  Variance  Percent 


Component 

1.380 

41.21% 

0.700 

22.75% 

MOS 

0.048 

1.43% 

0.046 

1.49% 

Component  x  MOS 

0.115 

3.43% 

0.163 

5.30% 

Rank 

0.014 

0.42% 

0.015 

0.49% 

Command 

0.013 

0.39% 

0.000 

0.00% 

Rank  x  Command 

0.000 

0.00% 

0.037 

1.20% 

Rank  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Command  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  Command  x  MOS 

0.044 

1.31% 

0.086 

2.79% 

Component  x  Rank 

0.000 

0.00% 

0.037 

1.20% 

Component  x  Command 

0.062 

1.85% 

0.000 

0.00% 

Comp  X  Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.064 

1.91% 

0.008 

0.26% 

Comp  X  Command  x  MOS 

0.236 

7.05% 

0.079 

2.57% 

Cp  X  Rk  x  Cd  X  M 

0.000 

0.00% 

0.018 

0.58% 

Rater(Rk  Cd  M) 

0.267 

7.97% 

0.346 

11.24% 

Component  x  Rater 

1.106 

33.02% 

1.542 

50.11% 

Total 

3.349 

100.00% 

aaaaaaaaa: 

3.077 

saaaaaass 

100.00% 

aaaaaaaa 

Another  way  to  assess  reliability  was  to  examine  fidelity 
coefficients,  defined  as  the  correlation  of  the  individual  rating 
profile  with  the  average  rating  profile  for  a  job.  We  correlated  a 
judge's  rating  profile  with  the  mean  rating  profile  for  the  judge's 
MOS.  Mean  fidelity  (correlation)  coefficients  of  the  four  rating 
types  for  each  MOS  are  summarized  in  Table  6a  to  6d.  The  fidelity 
coefficients  for  the  Task  Category  ratings  were  generally  higher  than 
those  for  Job  Activity.  For  the  Task  Category,  the  average  fidelity 
coefficients  for  the  four  ratings  ranged  from  .69  to  .79.  For  the 
Job  Activity,  the  mean  fidelity  coefficients  ranged  from  .55  to  .77. 
Correlations  between  a  judge's  profile  and  the  mean  profile  for  a 
different  job  (e.g.,  IIB  with  63B  mean)  averaged  .20  to  .30  points 
lower  than  member  fidelity  coefficients.  This  result  also  provided 
more  support  for  the  discriminability  of  job  description  among  the 
three  MOS. 
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Table  6a 

Mean  Fidelity  Coefficients  for  4  Ratings  with 
Frequency  of  Performance 


Task  Category  Job  Activity 

IIB  63B  71L  IIB  63B  71L 


MOS 

Rating  Type  (n=88) 

(n=50) 

{n-52) 

(n-88) 

(n=50) 

{n=52) 

Frequency 

0.76 

0.47 

0.28 

0.67 

0.30 

0.21 

Core  Technical 

0.75 

0.44 

0.25 

0.66 

0.27 

0.18 

IIB 

Gen.  Soldiering 

0.75 

0.46 

0.28 

0.66 

0.28 

0.25 

Overall 

0.75 

0.46 

0.26 

0.66 

0.27 

0.23 

Frequency 

0.49 

0.74 

0.35 

0.33 

0.62 

0.05 

Core  Technical 

0.43 

0.73 

0.35 

0.21 

0.59 

-0.02 

63B 

Gen.  Soldiering 

0.67 

0.66 

0.38 

0.57 

0.49 

0.17 

Overall 

0.59 

0.72 

0.36 

0.41 

0.60 

0.05 

Frequency 

0.27 

0.33 

0.79 

0.19 

0.03 

0.72 

Core  Technical 

0.18 

0.27 

0.77 

0.10 

-0.02 

0.69 

71L 

Gen.  Soldiering 

0.63 

0.48 

0.56 

0.49 

0.19 

0.58 

Overall 

0.47 

0.41 

0.73 

0.31 

0.06 

0.69 

Table  6b 

Mean  Fidelity  Coefficients  for  4  Ratings  with 
Importance  for  Core  Technical  Proficiency 

Task  Category  Job  Activity 


MOS 

IIB  63B  71L 

Rating  Type  (n=88)  (n=50)  (n=52) 

IIB 

(n*87) 

63B 

(n=50) 

71L 

(n=51) 

Frequency 

0.77 

0.38 

0.17 

0.64 

0.19 

0.11 

Core  Technical 

0.77 

0.35 

0.14 

0.65 

0.16 

0.09 

IIB 

Gen.  Soldiering 

0.77 

0.37 

0.17 

0.64 

0.18 

0.17 

Overall 

0.77 

0.37 

0.15 

0.65 

0.17 

0.14 

Frequency 

0.47 

0.67 

0.28 

0.28 

0.56 

-0.02 

Core  Technical 

0.41 

0.69 

0.29 

0.16 

0.59 

-0.04 

63B 

Gen.  Soldiering 

0.67 

0.57 

0.27 

0.55 

0.39 

0.08 

Overall 

0.58 

0.64 

0.26 

0.37 

0.54 

-0.03 

Frequency 

0.25 

0.31 

0.73 

0.16 

rO.Ol 

0.71 

Core  Technical 

0.15 

0.27 

0.75 

0.07 

-0.02 

0.74 

71L 

Gen.  Soldiering 

0.63 

0.40 

0.44 

0.46 

0.10 

0.53 

Overall 

0.46 

0.36 

0.64 

0.29 

0.00 

0.68 
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Table  6c 


Mean  Fidelity  Coefficients  for  4  Ratings  with 
Importance  for  General  Soldiering 


Task  Category  Job  Activity 

UB  63B  71L  IIB  63B  71L 


MOS 

Rating  Type  (n=88) 

(n=50) 

(n-52) 

(n-87) 

(n-50) 

{n=50) 

Frequency 

0.74 

0.64 

0.60 

0.61 

0.47 

0.45 

Core  Technical 

0.75 

0.62 

0.59 

0.62 

0.47 

0.45 

IIB 

Gen.  Soldiering 

0.75 

0.64 

0.61 

0.62 

0.48 

0.49 

Overall 

0.75 

0.63 

0.60 

0.62 

0.47 

0.47 

Frequency 

0.47 

0.65 

0.47 

0.27 

0.42 

0.19 

Core  Technical 

0.41 

0.60 

0.43 

0.17 

0.35 

0.09 

63B 

Gen.  Soldiering 

0.66 

0.73 

0.64 

0.54 

0.55 

0.41 

Overall 

0.58 

0.71 

0.57 

0.36 

0.48 

0.24 

Frequency 

0.27 

0.36 

0.52 

0.21 

0.14 

0.50 

Core  Technical 

0.17 

0.27 

0.43 

0.14 

0.07 

0.45 

71L 

Gen.  Soldiering 

0.63 

0.64 

0.72 

0.49 

0.37 

0.62 

Overall 

0.47 

0.52 

0.66 

0.33 

0.22 

0.57 

Table  6d 

Mean  Fidelity  Coefficients  for  4  Ratings  with 
Importance  for  Overall  Performance 

Task  Category  Job  Activity 


MOS 

Rating  Type  i 

IIB 

(n=88) 

638 

(n=50) 

71L 

(n=52) 

IIB 

{n=87) 

63B 

(n=50) 

71L 

(n=51) 

Frequency 

0.76 

0.57 

0.46 

0.64 

0.34 

0.31 

Core  Technical 

0.76 

0.55 

0.44 

0.65 

0.33 

0.30 

IIB 

Gen.  Soldiering 

0.76 

0.56 

0.47 

0.65 

0.33 

0.36 

Overall 

0.77 

0.56 

0.45 

0.65 

0.33 

0.34 

Frequency 

0.48 

0.72 

0.41 

0.28 

0.55 

0.07 

Core  Technical 

0.42 

0.69 

0.39 

0.17 

0.52 

-0.01 

63B 

Gen.  Soldiering 

0.67 

0.71 

0.53 

0.55 

0.51 

0.26 

Overall 

0.59 

0.74 

0.48 

0.37 

0.57 

0.10 

Frequency 

0.26 

0.34 

0.69 

0.20 

.0.04 

0.65 

Core  Technical 

0.16 

0.26 

0.63 

0.12 

-0.02 

0.61 

71L 

Gen.  Soldiering 

0.63 

0.57 

0.68 

0.49 

0.22 

0.62 

Overall 

0.47 

0.47 

0.74 

0.32 

0.08 

0.67 

(r 
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Comprehensiveness  of  Taxonomy.  At  the  end  of  the  instrument, 
judges  were  queried  about  how  completely  the  taxonomy  covered  the  job 
they  were  rating.  The  results  of  the  evaluation  are  found  in  Table 
7.  Overall,  the  judges  reported  that  the  Task  Category  model  covered 
69%  of  the  job  and  the  Job  Activity  instrument,  74%  of  the  job. 
Although  no  differences  were  found  for  judges'  responses  by  MOS,  some 
differences  were  found  by  rank  and  by  Command.  For  each  MOS,  NCOs 
indicated  that  the  instruments  provided  poorer  coverage  than  officers 
or  civilians.  Similarly,  FORSCOM  judges  reported  poorer  job  coverage 
than  DOTD  judges  for  both  instruments. 

Summary  and  Discussion 

A  number  of  conclusions  were  drawn  from  the  results.  First, 
reliability  appears  adequate,  with  the  possible  exception  of  NCOs 
using  the  Job  Activities  instrument.  Even  these  means  are  adequate 
across  a  number  of  different  judges.  Second,  discriminabil ity 
appears  good,  particularly  for  the  Job  Activity  instrument.  In 
addition,  coverage  appears  adequate.  While  the  absolute  meaning  of 
these  judgments  is  unclear,  they  do  indicate  slightly  greater 
coverage  by  the  Job  Activity  instrument. 

In  comparing  the  two  models,  there  was  no  clear  winner.  The 
greater  reliability  of  the  Task  Category  instrument  must  be  balanced 
against  the  lower  discriminabil ity  and  possibly  lower  coverage  that 
it  provides.  We  await  the  results  of  the  evaluation  of  the  overall 
models  through  comparison  with  empirical  results.  These  comparisons 
will  be  presented  in  Chapter  6. 
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Table  7 

Summary  of  Judges'  Evaluation  of  Instruments  for  Comprehensiveness 


What  percent  of  the  job  (you  are  rating)  is  covered  by  these  tasks/activities? 


IIB  63B  71L  All  Cases 

Percent  Percent  Percent  Percent 


MODEL 

n 

Mean 

SD 

n 

Mean 

SD 

n 

Mean 

SD 

n 

Mean 

SO 

Task 

70 

69.2 

31.8 

42 

69.5  28.5 

48 

69.0 

35.2 

160 

69.2 

31.9 

Activity 

73 

74.0 

28.5 

45 

76.8  26.7 

44 

72.5 

33.1 

162 

74.4 

29.2 

NCO 

Officer 

Civilian 

All  Cases 

Percent 

Percent 

Percent 

Percent 

MODEL 

n 

Mean 

SD 

n 

Mean 

SD 

n 

Mean 

SD 

n 

Mean 

SD 

Task 

84 

53.2 

31.9 

69 

88.9 

18.6 

7 

67.1 

28.1 

160 

69.2 

31.9 

Activity 

83 

61.0 

31.3 

70 

90.6 

14.8 

9 

72.2 

32.7 

162 

74.4 

29.2 

School 

Field 

All  Cases 

Percent 

Percent 

Percent 

MODEL 

n 

Mean  SD 

n 

Mean 

SO 

n 

Mean  SO 

Task 

46 

61.5  32.9 

114 

72.3 

31.0 

160 

69.2  31.9 

Activity 

49 

65.2  31.5 

113 

78.4 

27.4 

162 

74.4  29.2 

49 


REFERENCES 


Berliner,  D.  C.,  Angell ,  D.,  i  Shearer,  J.  W.  (1984).  Behaviors. 

measures,  and  instruments  for  performance  evaluation  in  simulated 
environments.  Paper  presented  at  a  symposium  and  workshop  on 
the  quantification  of  human  performance,  Albuquerque,  New  Mexico. 

Cunningham,  J.  W.,  Boese,  R.  R.,  Neeb,  R.  W.,  &  Pass,  J.  0.  (1983). 
Systematically  derived  work  dimensions:  Factor  analysis  of  the 
Occupation  Analyses  Inventory.  Journal  of  Applied  Psychology. 

232-252 

Fine,  S.  A.  (1962).  Functional  job  analysis  as  a  method  of  indirect 

validation:  a  study  in  synthetic  validation.  Unpublished  doctoral 
dissertation,  George  Washington  University,  Washington,  DC. 

Fine,  S.  A.  (1963).  Functional  Job  Analysis  (FJA)  as  a  method  of 
indirect  validation,.  American  Psychologist.  18,  438-445. 

Fine,  S.  A.  (1974).  Functional  job  analysis:  An  approach  to  a 

technology  for  manpower  planning.  Personnel  Journal .  W,  813- 
818. 

Fleishman,  E.  A.,  &  Quaintance,  M.  K.  (1984).  Taxonomies  of  human 

performance:  The  description  of  human  tasks.  Orlando:  Academic 
Press,  Inc. 

McCormick,  E.  J.,  Jeanneret,  P.  R.  &  Mecham,  R.  C.  (1972).  A  study  of 
job  characteristics  and  job  dimensions  as  based  on  the  Position 
Analysis  Questionnaire  (PAQ)  [Monograph].  Journal  of  AppI ied 
Psychology.  56,  347-368. 

Wing,  H.,  Peterson,  N.  G.,  &  Hoffman,  R.  G.  (1984).  Expert  .judgments 
of  predictor-criterion  validity  relationships.  Paper  presented 
at  the  annual  convention  of  the  American  Psychological 
Association,  Toronto. 

Yukl,  G.  A.  (1981).  Leadership  in  organizations.  New  Jersey: 
Prentice-Hall,  Inc. 


so 


CHAPTER  4: 


Development  of  an  Attribute  Taxonomy  and  Its 
Application  in  the  Formation  of  Synthetic  Validity  Composites 

Cynthia  K.  Owens-Kurtz  and  Norman  G.  Peterson 
Personnel  Decisions  Research  Institute,  Inc. 


SUMMARY 

This  chapter  focuses  on  the  development  of  an  attribute  taxonomy 
and  its  use  in  one  method  of  synthetic  validation,  in  which  the 
taxonomy  is  linked  to  whole  jobs,  or  to  a  few  job  factors,  by 
individuals  familiar  with  the  jobs. 

A  taxonomy  of  31  attributes  was  developed  and  applied  in  three 
sets  of  experiments  with  Army  Officers,  NCOs,  and  civilians.  Four 
judgment  methods  were  investigated: 

•  ratings  of  attribute  validity  for  five  job  performance 
areas 

•  ratings  of  attribute  importance  for  five  job  performance 
areas 

•  estimates  of  attribute  test  scores  for  individuals  in  the 
bottom,  middle,  and  top  thirds  on  job  performance 

•  rank  ordering  of  attributes  according  to  validity  and 
importance  for  overall  job  performance. 

The  five  job  performance  areas,  obtained  from  factor  analyses  of 
72  Army  criteria  (Wing,  Peterson,  &  Hoffman,  1984),  were:  Core 
Technical  Proficiency,  performance  on  tasks  central  to  the  specific 
job;  General  Soldiering  Proficiency,  performance  on  tasks  common  across 
all  Army  jobs;  Effort  and  Leadership;  Personal  Discipline;  and  Physical 
Fitness  and  Military  Bearing.  Subjects  were  least  confident  of  their 
judgments  in  the  score  estimation  method,  and  had  lowest  inter-rater 
agreement  in  this  method.  Results  indicate  that  soldiers  can  estimate 
the  importance  and  validity  of  attributes  for  five  job  performance 
areas,  and  can  rank  order  the  attributes  according  to  importance  and 
validity  for  overall  performance,  with  high  agreement. 

Validity  profiles  differed  between  jobs  for  the  Core  Technical 
component  of  soldiers'  jobs  and  for  overall  performance  ranking,  but 
not  for  components  common  across  jobs.  A  multi-trait  multi-method 
analysis,  treating  jobs  as  traits  and  validi^’y  and  importance  ratings 
as  methods,  found  high  convergent  validity  for  all  performance  areas 
and  overall  ranking;  discriminant  validities  averaged  .41  lower  for 
the  Core  Technical  component  and  overall  ranking  than  for  the  other 
performance  areas.  These  results  closely  parallel  prior  research 
regarding  the  validity  relationships  between  predictors  and  Army  job 
performance  areas  (Campbell,  1986). 
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CHAPTER  4 


Development  of  an  Attribute  Taxonomy  and  Its 
Application  in  the  Formation  of  Synthetic  Validity  Composites 

INTRODUCTION 

An  attribute  taxonomy  can  be  utilized  in  synthetic  validation  in 
two  ways.  First,  the  attribute  taxonomy  can  be  judgmental ly  linked 
to  job  components,  such  as  tasks  and  activities,  by  persons 
knowledgeable  about  measurement  of  human  abilities.  We  call  this 
version  of  the  synthetic  model  the  Psychologist  method.  Alternatively, 
the  attribute  taxonomy  can  be  linked  to  whole  jobs,  or  a  small  number 
of  broad  job  factors,  by  individuals  familiar  with  the  jobs,  and 
perhaps  not  as  familiar  with  human  abilities.  In  the  case  of  military 
jobs,  these  individuals  are  civilians  working  in  military  training  or 
curriculum  development,  non-commissioned  officers  (NCOs),  or  officers. 
We  have  called  this  version  of  the  synthetic  model  the  Soldier  method. 

In  the  Psychologist  method,  the  tasks  or  activities  act  as  an 
intervening  taxonomy  between  the  attributes  and  the  job  (Figure  1). 

In  contrast,  the  Soldier  method  has  no  intervening  taxonomy,  or  a 
very  small  one  consisting  of  a  few  broad  job  factors.  While  the 
Psychologist  method  links  attributes  to  job  tasks  or  activities  "once 
and  for  all,"  the  Soldier  method  requires  the  establishment  of  the 
attribute-job  link  for  each  job.  Because  they  are  unfamiliar  with 
attributes,  soldiers  may  have  difficulty  providing  attribute-job 
links.  This  chapter  will  focus  on  the  development  of  an  attribute 
taxonomy  and  use  of  the  taxonomy  in  the  Soldier  method.  The 
Psychologist  method  will  be  discussed  in  more  detail  in  chapter  5. 

DEVELOPMENT  OF  THE  ATTRIBUTE  METHODOLOGY 

We  describe  here  the  development  of  the  initial  attribute 
taxonomy,  the  development  of  judgment  methods,  the  pretesting  of  the 
taxonomy  and  methods,  and  revisions  to  the  judgment  methods  and 
taxonomy. 


Developing  the  Initial  Taxonomy 

The  initial  attribute  taxonomy  was  based  on  expert  judgment 
research  conducted  by  Wing,  Peterson,  and  Hoffman  (1984)  for  Army 
Project  A.  Project  A  is  a  multi-year  research  effort  designed  to 
improve  the  selection,  classification,  and  utilization  of  Army  enlisted 
personnel  (U.S.  Army  Research  Institute,  1983).  In  the  Wing  et  al . 
(1984)  research,  35  experts  in  personnel  psychology  estimated  the 
validity  of  53  predictor  variables,  or  constructs,  for  72  job  criteria. 
The  53  constructs  were  chosen  through  an  extensive  literature  search 
(Hough,  1988;  McHenry  &  Rose,  1988;  Toquam,  Corpe,  Dunnette,  &  Keyes, 
in  preparation).  Variables  in  three  broad  categories  were 
investigated:  cognitive  abilities,  noncognitive  attributes  such  as 
temperament  and  interests,  and  psychomotor  and  perceptual  abilities. 
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PSYCHOLOGIST  METHOD 


Attribute  1< 
Attribute  2 


Attribute  K 


>  Task  1  < 
Task  2  < 


Task  L 


Job  B 

Job  ZZZ 


SOLDIER  METHOD 

Attribute  1  < - ( - 1 - >  Job  A 

Attribute  2 

:  (Maybe  5>6  job 

taxons  here) 

« 

Attribute  K  < - j - 1 - >  Job  ZZZ 


Figure  1.  The  Psychologist  and  Soldier  Methods  for  Linking  Attributes 
to  jobs. 


Each  variable  considered  for  inclusion  was  evaluated  against  12 
factors,  such  as  Test  Fairness.  Based  on  the  estimated  validities 
provided  by  the. judges,  the  53  constructs  were  grouped  into  21 
clusters.  These  21  clusters  formed  the  basis  of  the  initial  taxonomy 
of  21  attributes  used  in  the  present  research. 

For  each  attribute,  we  wrote  a  definition  and  description  of 
individuals  high,  average,  and  low  on  the  attribute.  The  definitions 
and  descriptions  were  written  as  nontechnical ly  as  possible  to  insure 
understanding  by  persons  unfamiliar  with  testing  jargon.  These 
materials  were  reviewed  for  clarity  by  several  other  members  of  the 
project  staff. 

Developing  Initial  Methods  for  Collecting  Judgments 

We  developed  three  judgment  methods  initially:  validity  ratings, 
importance  ratings,  and  score  estimation.  In  the  validity  rating 
method,  soldiers  were  provided  with  a  brief  lecture  and  handouts 
describing  validity.  Soldiers  then  rated  the  validity  of  each 
attribute  for  job  performance.  In  the  importance  rating  method, 
soldiers  rated  the  importance  of  each  attribute  to  job  performance. 

The  score  estimation  method  required  soldiers  to  think  about 
individuals  who  were  in  the  lowest  one-third,  middle  one-third,  and 
highest  one-third  on  job  performance,  and  to  estimate  the  scores  of 
each  group  on  a  test  of  each  attribute. 

Pretests  and  Revisions 

Pretest  sample  and  method.  Pretests  were  conducted  with  NCOs 
and  officers  from  nine  Military  Occupational  Specialties  (MOS)  at 
three  sites  (N  -  202).  Several  key  questions  were  investigated  in 
the  Pretests:  the  feasibility  of  the  judgment  task,  the  optimal 
level  of  detail  for  performance  definitions,  the  number  of  performance 
areas,  and  the  clarity  of  instructions  and  definitions. 

A  subset  of  10  attributes  from  the  attribute  taxonomy  was 
administered  in  the  first  Pretest;  the  full  set  of  21  attributes  was 
administered  in  the  remaining  Pretests. 

The  performance  definition  and  number  of  performance  areas  were 
varied  in  the  first  Pretest  only.  Soldiers  were  provided  with  a 
short  definition  of  five  job  performance  areas,  a  detailed  definition 
of  five  job  performance  areas,  or  the  official  job  description  for 
their  MOS.  The  five  performance  areas  were  derived  from  factor 
analyses  of  Project  A  criterion  data  (Wing  et  al.,  1984).  The  areas 
are  Core  Technical  Proficiency,  General  Soldiering  Proficiency, 

Effort  and  Leadership,  Personal  Discipline,  and  Physical 
Fitness/Military  Bearing.  Core  Technical  Proficiency  includes  the 
proficiency  with  which  the  soldier  performs  the  tasks  that  are  central 
to  the  MOS.  These  central  tasks  differ  from  one  MOS  to  another.  The 
remaining  four  performance  areas  are  common  to  all  MOS.  Judgments 
were  made  for  the  five  performance  areas  or  for  overall  job 
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central  to  the  MOS.  These  central  tasks  differ  from  one  MOS  to 
another.  The  remaining  four  performance  areas  are  common  to  all  MOS. 
Judgments  were  made  for  the  five  performance  areas  or  for  overall  job 
performance.  Subjects  in  the  first  Pretest  indicated  a  preference 
for  the  detailed  definition  of  job  performance  supplemented  with  the 
official  job  description,  and  for  rating  all  five  performance  areas 
plus  overall  performance}  these  materials  were  used  for  the  remaining 
Pretests. 

We  administered  an  evaluation  questionnaire  at  the  end  of  each 
judgment  task.  The  questionnaire  assessed  the  clarity  of  the 
judgment  task  instructions,  difficulty  of  the  judgment  task,  and 
clarity  of  the  attribute  definitions. 

Pretest  results.  From  the  Pretest  debriefings  and  data,  we 
concluded  that  each  of  the  three  judgment  tasks  was  feasible. 
Interrater  agreement  reliabilities  were  computed  for  the  largest 
Pretest  (n  *  149).  Single-rater  reliabilities  were  lowest  for  the 
score  estimation  task,  and  were  higher  for  officers  than  for  NCOs. 

Responses  to  the  evaluation  questionnaire  indicated  that 
subjects  found  the  judgment  tasks  moderately  difficult;  the  score 
estimation  task  received  the  highest  mean  difficulty  rating. 

Subjects  who  completed  validity  or  importance  ratings  indicated 
greater  confidence  in  their  judgments  than  subjects  who  completed  the 
score  estimation  judgments.  Two  attributes,  spatial  ability  and 
closure,  were  described  as  especially  difficult  to  rate.  Overall, 
the  attribute  definitions  were  rated  as  clear. 

Revisions.  Based  on  the  low  reliability,  higher  difficulty 
ratings,  and  lower  confidence  ratings  of  the  score  estimation  method, 
it  was  dropped  from  further  investigation. 

Following  the  Pretests,  a  detailed  matching  of  the  synthetic 
validity  attribute  taxonomy  to  the  Wing  et  al.  (1984)  constructs 
identified  six  attributes  that  were  missing  from  the  synthetic 
validity  attribute  taxonomy..  The  six  attributes  were  added  to  the 
taxonomy.  A  review  of  predictor  tests  from  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB)  and  the  Project  A  Revised  Trial 
Battery  (Peterson,  1987)  identified  three  physical  ability  attributes 
and  one  psychomotor  attribute  not  included  in  either  Wing  et  al . 
(1984)  or  the  attribute  taxonomy.  Because  these  attributes  are 
believed  to  be  important  for  performance  on  some  Army  jobs,  they  were 
added  to  the  synthetic  validity  attribute  taxonomy.  Following  these 
revisions,  the  taxonomy  contained  31  attributes. 

APPLICATION  OF  THE  SOLDIER  METHOD 

We  report  here  on  the  first  two  iterations  in  the  application  of 
the  Soldier  method.  We  have  labeled  these  the  Pilot  Test  and  Phase 
I. 
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Pilot  Tests 


Pilot  Test  Sample  and  Method 

Validity  and  importance  ratings  of  the  31  attributes  for  the 
five  job  performance  areas  were  obtained  from  civilians,  NCOs,  and 
Officers  (N=127)  in  three  MOS:  Infantryman,  Mechanic,  and 
Administrative  Specialist.  Subjects  also  rank  ordered  the  31 
attributes  according  to  validity  and  importance  for  overall 
performance.  An  evaluation  questionnaire  was  administered  to  assess 
the  clarity  of  instructions,  difficulty  of  the  ranking  task,  and 
clarity  and  completeness  of  the  attribute  definitions. 

Pilot  Test  Results 


Because  the  number  of  civilians  in  the  study  was  extremely  low 
for  both  validity  and  importance  methods,  all  analyses  were  limited 
to  NCOS  and  Officers. 

Analyses  of  variance.  Due  to  a  small  sample  size  for  Mechanics 
for  the  importance  method,  analyses  of  variance  were  limited  to  the 
validity  ratings  for  the  Hve  performance  areas  and  validity  rankings 
for  overall  performance.  There  were  two  between- subjects  factors 
(MOS  and  Rater  Type)  and  one  within-subject  factor  (Attribute). 

Highly  significant  Attribute  effects  for  all  five  performance  areas 
and  for  the  rankings  were  found.  Thus,  the  attributes  did  receive 
different  ratings  and  rankings  when  averaged  across  all  three  MOS  and 
both  rater  types. 

The  Attribute  x  MOS  effect  was  highly  significant  for  the  Core 
Technical  Proficiency  performance  area  (Fgn  1950  *  8.49;  p<.01)  and 
the  overall  ranking  (F50  »  8.46;  p<.01),  and  much  less  significant 

for  the  Physical  Fitness/Mil itary  Bearing  area  (Fgg  1950  “  1*48; 
p<.05).  It  did  not  reach  significance  for  the  remaining  three 
performance  areas.  This  finding  accords  with  prior  expectations  and 
Project  A  empirical  validity  results  (Campbell,  1986);  there  are 
validity  differences  for  attributes  with  respect  to  the  Core  Technical 
component  of  soldiers'  jobs,  but  not  with  respect  to  the  "common" 
parts  of  soldiers'  jobs,  which  are  represented  by  the  other  four 
performance  areas. 

Small  but  statistically  significant  Attribute  x  Rater  Type 
effects  were  obtained  for  all  five  performance  areas,  but  not  for 
overall  ranking. 

Reliabilities.  Inter-rater  agreement  reliability  coefficients 
were  computed  for  validity  and  importance  ratings  and  rankings  using 
output  from  one-way  repeated  measures  ANOVA  to  estimate  mean  squares 
for  attributes  and  for  error;  rater  main  effect  was  thus  included  as 
part  of  the  error  term.  Reliabilities  were  calculated  for  each  MOS  x 
Performance  Area  x  Rater  Type  cell  where  the  cell  N  was  at  least  8. 
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Single-rater  reliabilities  range  from  .15  to  .61  (mean  »  .33)  for 
validity  ratings  and  rankings,  and  from  .25  to  .61  (mean  -  .41)  for 
importance  ratings  and  rankings.  Although  it  was  not  possible  to 
compare  Officer  and  NCO  ratings  for  all  MOS  and  both  methods,  the 
information  available  suggested  that  Officers  generally  have  greater 
agreement  than  do  NCOs. 

Multitrait  multimethod  analyses.  If  one  thinks  of  MOS  as  traits 
and  importance  and  validity  ratings  as  methods,  the  data  can  be 
analyzed  via  the  logic  of  the  multitrait,  multimethod  approach.  Table 
1  shows  the  results  for  the  five  performance  areas  separately,  the 
rankings,  and  all  five  areas  at  once.  Note  that  the  ratings  for  Core 
Technical  Proficiency  and  the  rankings  for  overall  performance  show 
very  high  mean  coefficients  for  convergent  validity  (between  method, 
within  MOS),  as  do  all  the  other  areas.  Note  further  that  the  mean 
coefficients  for  discriminant  validity  (between  method,  between  MOS 
and  within  method,  between  MOS)  are  on  average  about  .40  lower  for 
Core  Technical  Proficiency  and  rankings  than  for  the  other  four 
performance  areas.  This  indicates  that  different  profiles  of 
attributes  are  obtained  for  the  three  MOS  for  Core  Technical 
Proficiency  and  for  overall  ranking,  but  not  for  the  other  four 
performance  areas;  discriminant  validity  exists  for  these  two  but  not 
for  the  other  four.  These  are  precisely  the  results  we  would  expect 
based  on  prior  research  conducted  by  Project  A  regarding  the  validity 
relationships  between  predictors  and  these  five  performance  areas 
(Campbell,  1986). 

"Pol icv  capturing. "  Correlations  of  the  mean  validity  and  mean 
importance  rating  profiles  with  the  mean  validity  and  mean  importance 
ranking  profiles  were  computed  for  the  three  MOS  for  each  performance 
area  and  are  presented  in  Table  2.  These  analyses  represent  a  rough 
"policy  capturing"  of  the  rankings  of  the  attributes  for  overall 
performance.  That  is,  the  correlations  of  the  attribute  profiles  for 
each  performance  area  with  the  attribute  ranking  profile  indicates 
the  influence  of  each  job  performance  area  on  the  ranking  of 
attributes  for  overall  job  performance. 

For  Infantrymen,  Core  Technical  and  General  Soldiering  have 
nearly  equal  and  higher  correlations  with  the  ranking  profile  (r  » 

.75  to  .87)  than  do  the  other  three  areas,  although  the  other  three 
areas  do  have  relatively  high  correlations  with  the  ranking  profile 
(r  -  .47  to  .82).  For  Mechanics,  Core  Technical  has  high 
correlations  with  the  ranking  profile  (jr  «  .81  to  .92),  General 
Soldiering  has  moderate  correlations  with  ranking  (jr  -  .31  to  .47), 
and  the  remaining  three  areas  have  quite  low  correlations  with 
ranking  (r  «  00  to  .29).  For  Administrative  Specialists,  Core 
Technical  has  extremely  high  correlations  with  the  ranking  profile  (r 
«  .94  to  .95);  General  Soldiering,  Effort  and  Leadership,  and 
Personal  Discipline  have  moderate  correlations  with  ranking  (r  -  .53 
to  .67);  and  Fitness  and  Military  Bearing  has  very  small  negative 
correlations  with  ranking  (i  -  -.03  to  -.10). 
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Table  1.  Summary  of  Pilot  Test  Multi-Trait,  Multi-Method  Analyses:  Means  of  the  Convergen 
Validity  (Between  Method,  Within  MOS)  and  Discriminant  Validity  (Between  Method, 
Between  MOS  and  Within  Method,  Between  MOS)  Coefficients  for  Core  Technical,  Gene 
Soldiering,  Effort/Leadership,  Personal  Discipline,  and  Physical  Fltness/Mllltary 
Bearing  Performance  Areas,  Overall  Ranking,  and  All  Performance  Areas. 
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Table  2.  Correlations  of  Pilot  Test  Mean  Importance  and  Mean  Validity  Rankings  with  Mean 
Importance  and  Mean  Validity  Ratings  in  Each  of  the  Five  Performance  Areas 
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For  all  three  MOS,  the  validity  and  importance  ratings  show  the 
same  pattern  of  relationship  of  the  five  performance  areas  with  the 
rankings,  but  the  validity  ratings  consistently  correlate  lower  with 
the  rankings  than  do  the  importance  ratings.  This  could  indicate 
that  the  validity  ratings  are  less  susceptible  to  an  overall  "halo" 
effect  than  are  the  importance  ratings. 

Evaluations.  Evaluation  responses  indicate  that  soldiers  are 
moderately  confident  of  their  ratings  and  rankings.  As  in  the 
Pretests,  Spatial  Ability  and  Closure  were  identified  as  difficult 
attributes;  in  the  Pilot  Test,  several  Interest  attributes  were  also 
described  as  difficult.  Soldiers  stated  a  preference  for  using 
validity  or  importance  ratings  against  the  five  job  performance  areas 
over  using  rankings  to  "guide  selection  or  placement  of  soldiers," 
even  though  the  ranking  Judgments  were  less  complex  and  took  less 
time  to  complete. 


Phase  I 


Phase  I  Sample  and  Method 

Subjects  in  Phase  I  were  civilians,  NCOs,  and  Officers  (N  «  196) 
in  three  MOS:  Infantryman,  Mechanic,  and  Administrative  Specialist. 
Importance  ratings  and  rankings  were  eliminated  from  Phase  I  data 
collections.  The  evaluation  questionnaire  was  also  dropped. 

Validity  ratings  were  collected  for  only  two  of  the  five  performance 
areas,  Core  Technical  and  General  Soldiering  Proficiency.  In 
addition,  validities  for  five  common  job  tasks  (rated  most  important 
across  the  three  MOS  in  the  Pilot  Test)  and  five  specific  job  tasks 
(rated  most  important  within  the  MOS  in  the  Pilot  Test,  and  not 
common  tasks)  were  rated. 

Phase  I  Results 

Reliabilities.  Single-rater  reliabilities,  presented  in  Table 
3,  range  from  .19  to  .57  (mean  -  .33).  Officers  were  consistently 
more  reliable  than  NCOs,  with  differences  ranging  from  .02  to  .22 
(mean  difference  -  .12).  Mean  reliabilities  for  the  five  specific 
tasks  were  consistently  lower  than  mean  reliabilities  for  the  five 
common  tasks  (range  of  differences  -  .02  to  .19).  As  noted  in  the 
table,  at  least  15  raters  would  be  utilized  if  these  methods  were 
operationally  implemented,  yielding  reliabilities  greater  than  .80. 

"Policy  capturing."  Correlations  between  mean  validity  rating 
profiles  for  Core  Technical  and  General  Soldiering  and  mean  validity; 
ranking  profiles  for  overall  performance  appears  in  Table  4.  General 
Soldier  mean  profiles  for  each  MOS  are  highly  correlated  (range  ■  .92 
to  .97).  The  Core  Technical  mean  profile  for  Infantryman  is  highly 
correlated  with  General  Soldiering  mean  profiles  for  each  MOS  (range 
-  .93  to  .97).  The  Core  Technical  mean  profiles  for  Mechanic  and  for 
Administrative  Specialist  have  only  moderate  correlations  with  Core 
Technical  and  General  Soldiering  profiles  across  MOS  (range  -  .29  to 
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.58).  For  both  Mechanic  and  Administrative  Specialist,  mean  ranking 
proxies  are  highly  correlated  (r  =  .95  for  each)  with  Core  Technical 
mean  profiles,  and  only  moderately  correlated  with  General  Soldiering 
mean  profiles  (Mechanic,  r  =  .37;  Administrative  Specialist,  r  = 

.67).  These  results,  in  support  of  the  Pilot  Test  findings,  indicate 
that  soldiers  place  more  emphasis  on  core  job  tasks  than  common 
soldier  tasks  when  ranking  attribute  validities  for  overall  performance 

CONCLUSIONS 

Four  conclusions  follow  from  the  Pilot  Test  and  Phase  I  data. 
First,  Army  personnel  can  understand  the  materials  and  complete  the 
validity  and  importance  ratings  and  rankings.  Second,  inter-rater 
agreement  reliabilities  are  acceptable  and  indicate  that  15  raters 
should  supply  sufficiently  reliable  results  in  most  cases.  Third, 
Officers  have  higher  inter-rater  agreement  than  do  NCOs.  Fourth,  the 
pattern  of  judged  relationship  between  attributes  and  job  performance 
areas  corresponds  to  the  pattern  expected  and  the  pattern  found  in 
earlier  Army  Project  A  research  (Campbell,  1986). 
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CHAPTER  5: 


The  Use  of  Expert  Judges  to  Form  Synthetic 
Predictor  Composites  for  Army  Jobs 


Norman  6.  Peterson,  Rodney  R.  Rosse,  and  Cynthia  K.  Owens-Kurtz 
Personnel  Decisions  Research  Institute 


SUMMARY 


This  chapter  focuses  on  the  method  of  obtaining  expert  judgments 
of  the  validity  of  elements  in  an  attribute  taxonomy  for  predicting 
performance  on  elements  in  three  different  types  of  job  descriptor 
taxonomies  (tasks,  activities,  and  broad  job  performance  areas),  the 
reliability  of  those  judgments  for  various  subgroups  of  judges,  and 
the  method  of  using  the  validity  judgments  to  form  "synthetic" 
predictor  composites  for  purposes  of  predicting  performance  on  three 
military  jobs  (Infantryman,  Mechanic,  Administrative  Specialist). 

Sixty-nine  judges  estimated  validities  of  thirty-one  attributes 
for 

•  51  tasks 

•  29  activities 

•  5  broad  areas  of  performance  (for  one  of  the  3  military 
jobs) . 

Judges  completed  a  Background  Questionnaire  that  allowed  the  division 
of  the  sample  into  groups  that  stood  lower  and  higher  with  regard  to 
familiarity  with  the  military,  appropriate  psychological  training  and 
experience,  and  familiarity  with  research  completed  on  an  important, 
associated  military  research  project  (Project  A).  Reliability  of 
judgments  made  by  the  total  group  and  subgroups  was  assessed.  There 
appeared  to  be  very  little  difference  in  reliability  across  subgroups 
of  judges  or  across  the  type  of  validity  estimation  that  was  made. 

The  judgments  were  acceptably  reliable;  single-rater  reliabilities 
(based  on  total  group  estimates)  ranged  from  .42  to  .57  across  the 
type  of  validity  judgments.  The  judges'  validity  estimates  were 
combined  with  judgments  (by  military  NCOs  and  Officers)  of  the 
importance  of  the  job  descriptors  for  overall  job  performance  and 
with  information  about  the  covariance  of  measures  of  the  attributes 
to  form  "synthetic"  predictor  composite  scores.  This  method  of  forming 
synthetic  scores  is  described. 
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CHAPTER  5 


The  Use  of  Expert  Judges  to  Form  Synthetic 
Predictor  Composites  for  Army  Jobs 


INTRODUCTION 


The  previous  chapters  in  this  report  have  introduced  the  general 
purposes  for  using  synthetic  methods  of  forming  predictor  test 
composites  in  the  Army,  and  the  basic  taxonomies  that  are  candidates 
for  inclusion  in  the  synthetic  methodology.  In  this  chapter,  we 
present  our  recent  work  on  collecting  judgments  from  psychologists  to 
link  some  of  these  taxonomies  and  on  developing  of  models  for 
formulating  predictor  composite  scores  based  on  the  taxonomies, 
judgments,  and  logic  of  the  synthetic  methodology. 

Recall  that  one  of  the  primary  purposes  of  the  synthetic 
methodology  is  to  formulate  a  predictor  composite  score  from  a  battery 
of  predictor  tests  for  jobs  for  which  there  is  no  information  available 
about  predictor  tests  or  job  performance.  In  order  to  accomplish 
this  task,  taxonomies  must  be  available  to  describe  jobs  and 
predictors.  Previous  chapters  have  described  the  development  work  that 
has  resulted  in  four  taxonomies:  task  categories,  activities, 
performance  areas,  and  attributes.  As  already  presented,  the  first 
three  taxonomies  are  altarnative  methods  for  describing  "what  is 
done"  on  a  job,  while  the  fourth  taxonomy  is  a  method  for  describing 
the  abilities  required  to  perform  a  job,  or  "what  it  takes  to  do"  a 
job. 


In  our  research  we  have  thought  about  two  ways  of  using  these 
taxonomies  to  form  predictor  composite  scores,  the  "soldier"  method 
and  the  "psychologist"  method.  Recall  from  chapter  3  that  the 
"soldier"  method  requires  persons  who  are  expert  about  the  target 
jobs  (in  this  case,  Army  jobs,  thus  "soldiers")  to  make  judgments 
about  the  validity  of  attributes  for  the  job  as  a  whole,  or  for  a 
very  small  taxonomy  of  job-descriptive  elements.  In  the  "psychologist" 
method,  on  the  other  hand,  the  job  experts  make  judgments  about  the 
importance  of  elements  in  the  job-descriptive  taxonomy  for  effective 
performance  on  the  target  job.  Psychologists,  or  other  persons 
knowledgeable  about  the  domain  of  human  abilities  and  other  traits, 
make  the  judgments  that  link  the  job-descriptor  elements  to  the 
attributes. 

In  the  remainder  of  this  chapter,  we  describe: 

t  the  methods  and  materials  used  for  collecting  judgments 
from  psychologists  that  link  the  attribute  taxonomy  to  the 
three  job-descriptive  taxonomies 
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•  the  sample  of  psychologists  that  completed  the  judgments, 
together  with  some  attempts  to  quantify  their  degree  of 
familiarity  with  the  military  and  psychological  experience 
pertinent  to  the  kind  of  judgments  they  were  asked  to 
complete 

•  the  degree  of  reliability  of  the  judgments,  for  the  total 
sample  of  judges  and  for  various  sub-groups  of  judges 

•  the  way  in  which  we  formed  predictor  composite  scores  using 
the  psychologists'  judgments  together  with  judgments  from 
job  experts  and  information  about  the  covariances  of  the 
predictors 


METHODS  AND  MATERIALS  USED 
TO  COLLECT  PSYCHOLOGISTS'  JUDGMENTS 

Method 

The  method  employed  to  collect  the  judgments  linking  the 
attributes  to  the  task  category,  activity,  and  performance  area 
taxonomies  was  very  similar  to  the  method  used  by  Wing,  Peterson,  & 
Hoffman  (1984).  Judges  were  supplied  with  definitions  of  the 
attributes  and  definitions  of  job  descriptors  (the  task  categories, 
activities,  and  performance  areas).  Judges  were  then  asked  to  estimate 
the  validity  of  each  attribute  for  each  job  descriptor,  using  a 
nine-point  scale  with  each  scale  point  anchored  by  a  ten-point  range 
of  correlation  coefficient  values.  Their  estimates  were  entered  on 
forms  prepared  specifically  for  the  estimation  task.  Instructions 
were  written  to  guide  the  judges  through  the  process  of  making  their 
estimates.  (For  those  interested,  copies  of  the  rating  materials  and 
instructions  are  available  from  the  senior  author.)  Judges'  estimates 
were  independently  completed  at  their  own  pace,  without  supervision. 

Materials 

In  this  research  effort,  we  wished  to  obtain  judgments  linking 
the  31  attributes  to 

f  96  task  categories 

•  53  activities 

§  5  job  performance  areas  in  each  of  3  Army  jobs  (Infantryman, 

Mechanic,  and  Administrative  Specialist). 

We  wished  to  make  several  comparisons  of  the  uses  of  the  different 
judgments  which  argued  for  the  use  of  the  same  judges  for  linking  all 
the  taxonomies.  It  was  not  feasible,  however,  to  ask  each  judge  to 
make  such  a  large  number  of  judgments  (a  total  of  5,084). 
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Therefore,  we  divided  both  the  task  category  taxonomy  and  the 
activity  taxonomy  into  two  "parallel"  forms  by  placing  odd-numbered 
items  on  one  form  and  even-numbered  items  on  the  other  form.  (There 
were  some  minor  exceptions  to  this  rule  when  we  wished  to  more  evenly 
balance  the  content  of  the  two  forms).  In  addition,  we  placed  a  few 
items  on  both  forms  (there  were  six  common  items  for  the  two  task 
category  forms  and  four  common  items  for  the  two  activity  forms)  in 
order  to  provide  a  crude  check  on  the  comparability  of  the  general 
level  of  judgments  provided  by  the  groups  responding  to  the  two 
forms.  Also,  we  asked  each  judge  to  provide  judgments  linking  the 
attributes  to  only  one  of  the  three  Army  jobs,  rather  than  all  three. 
Each  judge  was  asked  to  provide  2697  judgments,  a  little  more  than 
half  of  the  number  required  to  complete  judgments  for  the  full  set  of 
taxonomies. 

We  are  also  interested  in  the  effects  of  judge  characteristics 
on  the  reliability  and  accuracy  of  the  estimates  they  provide.  In  an 
attempt  to  measure  appropriate  judge  characteristics,  we  prepared  a 
Background  Questionnaire  that  we  asked  each  judge  to  complete.  The 
questionnaire  was  designed  to  measure  two  primary  characteristics: 
degree  of  training/experience  in  relevant  psychological  content  areas 
and  degree  of  familiarity  or  knowledge  about  military  occupations. 

In  addition,  we  queried  judges  about  their  degree  of  familiarity  with 
the  Army's  Project  A,  because  we  make  much  use  on  this  project  of 
information  obtained  from  Project  A  research. 

Psychological  training/experience  was  measured  by  asking  judges 
to  indicate  their  familiarity  with  each  of  15  content  areas,  called 
tasks,  that  we  thought  covered  the  areas  critical  to  personnel 
selection  and  test  validation  endeavors  (some  examples  are  job 
analysis,  content  validation,  design  or  development  of  cognitive 
tests,  research  on  the  relationship  of  cognitive  tests  to  other 
variables).  The  judges  indicated  their  familiarity  by  responding 
"yes"  or  "no"  to  seven  descriptive  anchors  for  each  of  the  fifteen 
tasks.  The  descriptive  anchors  were: 

•  heard  about  this  task  in  undergraduate  course(s)  or  other 
general  sources 

•  studied  this  task  in  graduate  course(s),  or  studied  in 

depth  on  my  own 

•  performed  parts  of  this  task  under  supervision 

•  performed  this  task  without  supervision 

•  supervised  others  performing  this  task 

•  taught  this  task  to  others 

f  wrote  a  scholarly  article  or  book  about  this  task 
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To  measure  familiarity  with  the  military,  we  asked  judges  if 
they  had  learned  about  the  military  through  close  family  members,  if 
they  had  done  consulting  work  with  the  military,  or  if  they  had 
served  in  the  military.  We  also  asked  judges  about  their  familiarity 
with  15  job  tasks;  5  tasks  central  to  each  of  the  3  focal  jobs 
(Infantryman,  Mechanic,  Administrative  Specialist)  in  this  phase  of  the 
research. 

In  summary,  the  judges  received  the  following  materials: 

•  definitions  of  31  attributes 

•  Form  A  or  Form  B  of  task  category  definitions  (51  tasks) 

§  Form  A  or  Form  B  of  activity  definitions  (28  or  29  activities 

•  definitions  of  five  job  performance  areas  for  one  job 

•  answer  sheets  for  recording  validity  estimates 

•  background  Questionnaire 

•  instructions  for  completing  the  estimates  and  returning 

materials 

The  judges  were  instructed  to  complete  the  Background 
Questionnaire  first,  then  to  estimate  the  validities  for,  in  order, 
task  categories,  activities,  and  performance  areas. 


COLLECTION  OF  PSYCHOLOGISTS'  JUDGMENTS 
Selection  of  the  Sample 

The  target  population  from  which  we  obtained  estimates  included 
four  subgroups: 

e  highly  knowledgeable,  experienced  researchers; 

•  knowledgeable,  experienced  researchers; 

•  novice  researchers  in  the  process  of  becoming  educated  and 

experienced  with  regard  to  personnel  psychology;  and 

•  military  experts. 

The  military  experts  provided  ratings  in  the  Pretest,  Pilot  Test,  and 
Phase  I  rounds  of  the  project.  Previous  chapters  in  this  report 
discuss  the  military  experts'  judgments  and  results  (Chi a,  Hoffman, 
Campbell,  Szenas,  &  Crafts,  in  preparation;  Owens-Kurtz  i  Peterson, 
in  preparation).  Nominations  for  researchers  outside  the  contracting 
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organizations  who  qualified  for  groups  1  and  2  were  elicited  from  a 
highly  respected,  experienced,  visible  researcher  in  the  area  of 
selection  and  validation  and  from  a  younger,  recent  Ph.D.  with  research 
experience  in  selection  and  validation.  Those  individuals  who  appeared 
on  both  lists  were  included  in  the  sample;  the  remaining  names  were 
reviewed  by  key  members  of  the  research  team  and  a  subset  was  chosen 
for  inclusion.  The  final  sample  of  outside  experts  contained  40 
individuals. 

Each  contracting  organization  supplied  a  list  of  individuals 
within  the  organization  who  had  time  available  and  who  qualified  for 
groups  1,  2,  or  3.  Contractor  staff  who  fell  into  groups  1  and  2 
were  primarily  project  directors,  research  scientists,  senior 
researchers,  and  research  associates;  staff  in  group  3  were  primarily 
research  assistants  and  junior  researchers.  A  total  of  55  contractor 
staff  members  were  identified. 


Method 

Materials  were  mailed  to  targeted  individuals  with  a  cover 
letter  which  entreated  them  to  complete  the  judgments  and  set  a 
deadline  for  receipt  of  the  completed  forms.  For  outside  experts, 
the  cover  letter  also  explained  that  an  honorarium  would  be  paid  to 
those  who  completed  the  judgments. 

Final  Sample 

Completed  judgments  were  received  from  23  of  the  40  outside 
experts  solicited  (57.5%).  These  outside  experts  can  be  divided  into 
five  groups;  members  of  the  Scientific  Advisory  Committee  for  Project 
A  (N  =  4);  past-presidents  of  American  Psychological  Association 
(APA)  Division  14  (N  -  5);  APA  Fellows  (N  =  6);  APA  Members  (N  =  6); 
and  other  (N  =  2).  Forty-six  of  the  55  contractor  staff  members 
identified  as  potential  judges  completed  the  judgments  by  the  deadline 
(83.6%);  one  individual's  judgments  were  received  too  late  for 
inclusion  in  analyses.  The  total  sample  included  judgments  from  69 
individuals. 


ANALYSES  OF  PSYCHOLOGISTS'  JUDGMENTS 
Background  Questionnaire 

Background  Questionnaire  responses  were  analyzed  to  form  groups 
of  judges  based  on  psychological  experience,  familiarity  with  the 
military,  intersection  of  experience  and  familiarity,  and  familiarity 
with  Army  Project  A. 

Psychological  Experience  Scores 

Items  in  Section  II  of  the  Background  Questionnaire  assessed  the 
judges'  experience  in  personnel  psychology.  Each  item  required  a 
"no"  (0)  or  "yes"  (1)  response  to  seven  descriptive  anchors 
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representing  varying  degrees  of  experience.  The  item  score  equaled 
the  sum  of  the  anchor  responses.  A  second  scoring  method  was  examined, 
in  which  each  anchor  response  was  multiplied  by  a  weight  from  1  to  5 
representing  the  degree  of  experience  it  described;  the  sum  of  the 
anchor  scores  equaled  the  item  score.  Internal -consistency  reliability 
(coefficient  alpha)  was  high  for  both  scoring  schemes  for  each  item 
(range  =  .70  to  .85).  The  weighted  scoring  scheme  was  used  in  all 
subsequent  analyses.  Possible  scores  ranged  from  0  to  360;  obtained 
scores  ranged  from  34  to  336,  with  a  mean  of  143.75  and  standard 
deviation  of  78.24. 

Psychological  Experience  Groups 

The  distribution  of  psychological  experience  scores  was  examined 
to  determine  any  natural  breaking  points  at  which  the  judges  could  be 
separated  into  high  and  low  experience  groups.  A  break  occurred 
between  scores  of  154  and  173.  Judges  scoring  154  or  lower  formed 
the  low  experience  group  (N  =  44);  judges  scoring  173  or  higher 
formed  the  high  experience  group  (N  =  25). 

Mil itarv  Famil iaritv 

Section  I,  item  4  of  the  Background  Questionnaire  assessed  the 
judges'  experience  with  or  exposure  to  the  military.  Points  were 
awarded  for  exposure  through  a  spouse  or  family  member,  through  work 
as  a  consultant  with  military  organizations,  and  through  service  in 
the  military.  A  total  of  19  points  was  possible;  scores  obtained 
ranged  from  0  to  11  (mean  »  4.42;  standard  deviation  =  2.65). 

Items  in  Section  III  of  the  Background  Questionnaire  assessed 
the  judges'  familiarity  with  five  job  tasks  from  each  of  the  three 
MOS  under  investigation:  Infantryman,  Mechanic,  and  Administrative 
Specialist.  Judges  responded  0  (unfamiliar),  1  (somewhat  familiar), 
or  2  (very  familiar)  for  each  item.  A  familiarity  score  was  computed 
for  each  MOS  by  summing  the  responses  to  the  five  items  for  that  MOS. 
Internal -consistency  reliability  (coefficient  alpha)  for  Infantryman 
familiarity  was  .80;  for  Mechanic  familiarity,  .77;  and  for 
Administrative  Specialist  familiarity,  .73.  A  total  familiarity 
score  was  also  computed  by  summing  responses  to  all  15  items  in 
Section  III.  Internal -consi stency  reliability  for  total  familiarity 
score  was  high  (alpha  »  .85). 

Military  Familiarity  Groups 

Military  experience  scores  and  Infantryman  familiarity  scores, 
which  correlated  .60,  were  summed  to  form  a  military  familiarity 
score.  Possible  scores  ranged  from  0  to  34;  obtained  scores  ranged 
from  0  to  20  (mean  -  9.12,  standard  deviation  ■  4.64).  Judges  scoring 
8  or  lower  formed  the  low  military  familiarity  group  (N  -  35);  judges 
scoring  9  or  higher  formed  the  high  military  familiarity  group  (N  » 
34). 
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Intersection  of  Experience  and  Familiarity  Groups 


The  intersection  of  psychological  experience  and  military 
familiarity  groups  was  examined.  Judges  in  quadrant  1  were  low  on 
both  experience  and  familiarity  (N  =  27).  Judges  in  quadrant  2  were 
in  the  low  experience  and  high  familiarity  groups  (N  =  17).  Judges 
in  quadrant  3  were  in  the  high  experience  and  low  familiarity  groups 
(N  =  8).  Judges  in  quadrant  4  were  high  on  both  experience  and 
familiarity  (N  =  17). 

Project  A  Familiarity  Groups 

Section  I,  item  5  assessed  familiarity  with  Army  Project  A. 

Scores  ranged  from  1  to  5  with  a  mean  of  3.20  and  a  standard  deviation 
of  1.29.  Judges  scoring  1  or  2  (never  heard  of  Project  A  or  heard  a 
presentation  of  results)  formed  the  low  Project  A  familiarity  group 
(N  =  27).  Judges  scoring  3  (occasionally  worked  on  Project  A)  formed 
the  moderate  Project  A  familiarity  group  (N  *  16).  Judges  scoring  4 
or  5  (worked  on  Project  A  often  or  very  freque'^tly)  formed  the  high 
Project  A  familiarity  group  (N  =  26). 


Reliability  Analyses 

We  performed  analyses  to  estimate  the  reliability  of  the  judges' 
estimates  of  validities.  Separate  analyses  were  performed  for  Forms 
A  and  B  of  the  task  category  and  activity  xonomies  and  for  the 
three  military  jobs- -Infantryman,  Mechanic,  and  Administrative 
Specialist.  Analyses  of  variance  were  performed  to  provide  mean 
squares  for  computing  intraclass  coefficients.  We  were  interested  in 
the  reliability  of  the  estimates  of  validity  of  attributes  for  job 
descriptors,  represented  in  the  Analysis  of  Variance  (ANOVA)  as  the 
Descriptor  x  Attribute  interaction.  Table  1  shows  the  ANOVA  table 
for  the  analysis  of  validity  estimates  when  the  31  attributes  were 
rated  against  the  51  task  categories  found  on  Form  A.  All  the  judges 
who  completed  that  rating  task  were  included  in  the  analysis,  in  this 
case,  35  raters.  Also  shown  in  Table  1  are  the  intraclass  coefficients 
for  the  single  rater  case,  the  10,  20,  and  30  case,  and  the  coefficient 
appropriate  for  tha  number  of  raters  in  the  analysis. 

Table  2  shows  the  single-rater  reliabilities  for  the  total 
sample  of  judges  available  for  each  rating  task  and  for  several 
sub-groupings  of  the  total  sample  of  available  judges:  those  with  low 
and  high  scores  on  Military  Familiarity,  those  with  low  and  high 
scores  on  Psychological  Experience,  those  in  the  quadrants  formed  by 
the  intersection  of  these  two  groups,  and  those  with  low,  moderate, 
and  high  familiarity  with  Project  A.  In  Table  2,  MOS  IIB  - 
Infantryman,  MOS  63B  »  Mechanic,  and  MOS  7IL  -  Administrative 
Specialist. 

Note  first  that  the  single-rater  reliabilities  for  the  total 
available  raters  are  all  at  acceptably  high  levels,  ranging  from  a 
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low  value  of  .42  for  the  Infantryman  (IIB)  and  a  high  value  of  .57 
for  Form  A  of  the  activities  taxonomy. 

Table  1 

Sources  of  Variance,  Sums  of  Squares,  Degrees  of  Freedom,  Mean  Squares 
and  Intraclass  CoefHcients  for  Psychologists'  (N=35)  Estimates  of 
Validity  of  31  Attributes  for  51  Task  Categories  (Form  A) 


Source  of  Variance 

Sum  of  Squares 

DF 

Mean  Square 

Descriptor 

3741.549 

50 

74.831 

Attribute 

24538.641. 

30 

817.955 

Descriptor  x  Attribute 

43128.552 

1500 

28.752 

Rater 

14362.916 

34 

422.439 

Rater  x  Descriptor 

4885.012 

1700 

2.874 

Rater  x  Attribute 

17109.906 

1020 

16.774 

Rater  x  Desc.  x  Att. 

42134.966 

51000 

0.826 

The  intra-class  coefficients  for  the  vector 
of  attribute  x  descriptor  ratings  are: 

Single  rater:  0.491 
Mean  of  10  raters:  0.906 
Mean  of  20  raters:  0.951 
Mean  of  30  raters:  0.967 
Mean  of  all  raters  (35)  for  this  problem:  0.971 
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Table  2. 


Estimates  of  Single-Rater  Reliability  (Intraclass  Coefficients)  for 
Synthetic  Validity  Expert  Judgments,  By  MOS,  Task  Category  Form, 
Activity  Form,  Military  Familiarity,  Psychological  Experience, 
Project  A  Familiarity,  and  Total  Groups. 


Group 

IIB 

N  r 

N 

MOS 

63B 

r 

N 

71L 

r 

A 

N 

Tasks 

B 

r  N 

r 

N 

Activities 
A  B 

r  N 

r 

Low  Military 

Familiarity  (MF) 

13  .492 

11 

.522 

11 

.568 

18 

.505 

17 

.486 

18 

.588 

17 

.557 

High  Mi  1 i tary 

11  .333 

11 

.570 

12 

.479 

17 

.471 

17 

.456 

17 

.544 

17 

.562 

Familiarity 

Low  Psychological 

Experience  (PE) 

17  .437 

15 

.557 

12 

.561 

20 

24 

.460 

20 

.578 

24 

.550 

High  Psychological 

Experience 

7  .380 

7 

.557 

11 

.475 

15 

.482 

10 

.484 

15 

.559 

10 

.569 

Low  PE,  Low  MF 

11  .491 

9 

.546 

7 

.563 

13 

.506 

14 

.486 

13 

.573 

14 

.556 

Low  PE,  High  MF 

6  .307 

6 

.571 

5 

.556 

7 

.502 

10 

.430 

7 

.588 

10 

.544 

High  PE,  Low  MF 

2  .413 

2 

.421 

4 

.583  i 

5 

.519 

3 

.442 

5 

.625 

3 

.539 

High  PE,  High  MF 

5  .346 

5 

.588 

7 

.419  : 

10 

.454 

7 

.483 

10 

.518 

7 

.579 

Low  Project  A 

Famil iarity 

10  .375 

7 

.500 

10 

.440 

14 

.464 

13 

.466 

14 

.510 

13 

.540 

Moderate  Project  A 

Familiarity 

5  .454 

8 

.537 

3 

.506 

6 

.485 

10 

.538 

6 

.620 

10 

.608 

High  Project  A 

Familiarity 

9  .462 

7 

.591 

10 

.581 

15 

.521 

11 

.412 

15 

.605 

11 

.527 

Total 

24  .424 

22 

.545 

23 

.519 

35 

.491 

34 

.469 

35 

.566 

34 

.558 
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Our  hypotheses  about  the  subgroups  of  raters  were  that  those 
judges  with  greater  military  familiarity  and  more  psychological 
experience  should  be  "better"  judges,  and  thus  should  evidence  greater 
agreement  and  higher  validity  when  their  estimates  are  used  to  form 
prediction  equations.  We  hypothesized  a  similar  effect  with  regard 
to  Project  A  familiarity. 

Examination  of  Table  2  reveals  little  support  for  these 
hypotheses,  at  least  with  regard  to  rating  reliability.  Raters  high 
on  the  subgrouping  variables  sometimes  had  higher  reliability 
coefficients,  and  sometimes  had  lower  coefficients.  In  general,  the 
coefficients  are  not  that  different.  Of  the  84  coefficients,  only  9 
are  outside  the  range  from  .40  to  .60.  Although  we  have  performed  no 
statistical  tests,  it  seems  reasonably  safe  to  assume  that  sampling 
error  accounts  for  almost  all  of  the  variance  in  the  coefficients  in 
this  table. 


78 


REFERENCES 


Chia,  W.  J.,  Hoffman,  R.  G.,  Campbell,  J.  P.,  Szenas,  P.  L., 

Crafts,  J.  J.  (in  preparation).  Analysis  of  job  components:  The 
development  and  evaluation  of  alternative  methods.  In  L.  L. 

Wise,  J.  M.  Arabian,  W.  J.  Chia,  &  P.  L.  Szenas  (Eds.)  Army 
Synthetic  Validity  Project:  Report  of  Phase  I  results. 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behayioral 
and  Social  Sciences. 

Owens-Kurtz,  C.  K.,  Peterson,  N.  P.  (in  preparation).  Development  of 
an  Attribute  taxonomy  and  its  application  in  the  formation  of 
synthetic  validity  composites.  In  L.  L.  Wise,  J.  M.  Arabian,  W. 
J.  Chia,  &  P.  L.  Szenas  (Eds.)  Army  Synthetic  Validity  Project: 
Report  of  Phase  I  results.  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

Wing,  H.,  Peterson,  N.  G.,  &  Hoffman,  R.  G.  (1984).  Expert  judgments 
of  predictor-criterion  validity  relationships.  Paper  presented 
at  the  annual  convention  of  the  American  Psychological 
Association,  Toronto. 


^  rJ' 


79 


CHAPTER  6: 


Comparative  Analyses  of  Empirical  and  Synthetic  Job  Performance 

Prediction  Equations 

Lauress  L.  Wise 

American  Institutes  for  Research 

Norman  G.  Peterson  and  Rodney  R.  Rosse 
Personnel  Decisions  Research  Inc. 

John  P.  Campbell 

University  of  Minnesota  &  HumRRO 


SUMMARY 

In  this  chapter,  we  describe  the  empirical  derivation  of 
prediction  equations  for  the  three  jobs  in  Phase  I  of  our  study  and 
compare  these  equations  to  alternative  forms  of  the  synthetically 
derived  equations.  Two  major  comparisons  are  reported.  The  first  is 
between  the  different  job  component  models.  All  three  models  yielded 
prediction  equations  with  high  alidities  for  each  of  the  three  jobs. 
The  Attribute  Model  yielded  slightly  higher  validities  and  the 
Activities  Model  yielded  slightly  lower  validities  in  comparison  to 
the  Job  Task  Model.  None  of  the  models  discriminated  between  the 
three  jobs  very  well.  The  Attribute  Models  yielded  slightly  greater 
discrimination  and  the  Job  Task  Model  yielded  slightly  less 
discrimination  in  comparison  to  the  Activities  Model. 

The  relatively  low  levels  of  discrimination  prompted  further 
investigation  of  the  assumptions  used  in  establishing  the  attribute 
weights  for  predicting  performance  on  each  job  component  and  the 
component  weights  used  in  combining  the  prediction  equations.  The 
second  set  of  comparisons  focused  on  alternative  procedures  for 
computing  these  weights.  The  results  showed  that  alternative 
attribute-by-component  weights  led  to  significant  differences  in 
absolute  validity  and  only  minor  changes  in  discriminant  validity 
with  a  unit  weighting  approach  giving  the  best  results.  An  alternative 
scaling  of  the  component-by-job  weights  did  lead  to  significant 
improvement  in  the  discriminant  validity  of  the  synthetic  equations 
with  little  or  no  change  in  the  absolute  validities. 

The  results  indicate  a  high  degree  of  validity  and,  with 
appropriate  adjustments  to  the  attribute  and  component  weights, 
adequate  levels  of  discriminability  for  the  synthetically  derived 
predictions.  The  exploratory  analyses  of  alternative  weight 
adjustments  and  the  very  limited  number  of  jobs  suggest  the  strong 
need  for  further  confirmatory  analyses.  Such  analyses  are  planned  as 
parts  of  Phase  II  and  Phase  III  of  the  current  project. 
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CHAPTER  6: 


Comparative  Analyses  of  Empirical  and  Synthetic  Job  Performance 

Prediction  Equations 


In  this  chapter,  we  describe  the  results  of  our  preliminary 
evaluation  of  the  synthetically  derived  prediction  equations  for 
three  Army  jobs:  Infantryman,  Vehicle  Mechanic,  and  Administrative 
Specialist.  Two  general  criteria  are  used  in  these  evaluations.  The 
first  is  the  absolute  validity  of  the  predictions  for  each  job.  Data 
from  Project  A  were  used  to  obtain  empirical  estimates  of  these 
validities.  The  second  criterion  is  differential  validity.  To  what 
extent  are  the  prediction  composites  for  each  job  more  valid  for  that 
job  than  for  the  other  jobs? 

We  also  examined  alternative  wrinkles  in  the  methods  used  for 
forming  the  synthetic  prediction  composites.  First,  we  examined  the 
methods  used  for  obtaining  the  attribute  weights  used  in  predicting 
performance  on  each  of  the  job  components:  Then,  we  also  examined 
the  method  used  for  estimating  the  job  component  weights  used  in 
combining  the  component  prediction  t  ^.ations. 

Data 


The  data  used  in  these  analyses  were  taken  from  the  Project  A 
Concurrent  Validation.  The  O'-erall  data  set  included  predictor  and 
job  performance  measures  collected  on  soldiers  in  19  different  jobs. 
Three  of  these  jobs  were  the  focus  of  the  Phase  I  synthetic  validation 
efforts:  118  -  Infantryman,  638  -  Vehicle  Mechanic,  and  71L  - 
Administrative  Specialist. 

The  individual  predictor  measures  included  in  the  Project  A 
battery  have  been  described  in  detail  by  Peterson  et  al .  (Peterson, 
Hough,  Dunnette,  Rosse,  Houston,  &  Toquam,  1987).  Owens-Kurtz  and 
Peterson  above  describe  the  development  of  specific  measures  of  26  of 
the  predictory  taxonomy  elements  from  scales  in  the  Project  A  battery. 

Wise,  Campbell,  McHenry,  and  Hanser  (1986),  and  Campbell,  McHenry, 
&  Wise  (1987),  have  described  the  identification  and  measurement  of 
five  job  performance  constructs  of  interest  to  the  Army:  job-specific 
proficiency,  general  soldiering  proficiency,  effort  and  leadership, 
personal  discipline,  and  physical  fitness  and  military  bearing. 

Wise,  Campbell,  and  Peterson  (1987)  showed  that  the  same  predictor 
measures  are  optimal  for  a  wide  range  of  jobs  in  predicting  all  but 
the  first  of  these  constructs.  Significant  differences  across  jobs 
were  found  in  the  predictors  of  job-specific  proficiency. 

To  a  large  extent,  the  Army  uses  separate  measures  to  screen 
applicants  who  might  be  deficient  in  different  areas  of  performance. 
The  current  project  is  most  closely  focused  on  the  development  of 
prediction  composites  for  job-specific  aspects  of  performance.  For 
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this  reason,  the  Core  Technical  Proficiency  (CTP)  measure  was  used  as 
the  primary  performance  criterion  in  the  analyses  described  below. 

The  Concurrent  Validation  sample  Included  491  infantrymen,  478 
mechanics,  and  427  clerks  with  complete  data  on  the  predictor  and 
criterion  measures.  These  samples  differed  somewhat  in  terms  of  the 
heterogeneity  and  mean  levels  of  the  predictor  scores.  Also,  because 
all  were  selected  job  incumbents,  they  had  higher  and  less  variable 
predictor  scores  in  comparison  to  the  overall  pool  from  which 
applicants  are  drawn.  Common  practice  has  been  to  use  a  multivariate 
correction  to  adjust  covariances  and  correlations  for  differences  in 
heterogeneity  (Lord  &  Novick,  1968).  The  1980  Youth  Population 
sample  to  which  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  was  administered  is  used  as  the  target  population.  This 
procedure  corrects  for  effects  of  restriction  in  range  due  to 
explicit  selection  on  the  subtests  of  the  ASVAB. 

The  Concurrent  Validation  samples  were  further  restricted  by 
self-selection  into  each  occupational  specialty  and  by  attrition 
after  initial  enlistment.  We  used  a  two-step  procedure  to  adjust  for 
range  restriction  due  to  both  sources  of  selection.  First,  we 
estimated  the  covariance  of  the  26  predictor  attribute  measures  for 
the  entire  Concurrent  Validation  sample  (7,055  cases  with  complete 
data)  and  adjusted  these  covariances  for  differences  between  the 
Concurrent  Validation  (CV)  sample  and  the  Youth  Population  in  the 
covariances  of  the  ASVAB  subtests.  This  provided  us  with  estimates 
of  what  the  covariances  among  the  attribute  measures  would  have  been 
for  the  Youth  Population  had  all  of  the  Project  A  predictor  measures 
been  administered  to  them. 

Second,  we  computed  covariances  for  each  of  the  three 
job-specific  samples  that  included  the  26  predictors  plus  the  five 
criterion  construct  scores.  We  then  adjusted  these  covariances  for 
differences  between  the  job  specific  sample  and  the  estimated  Youth 
Population  covariances.  Table  1  shows  the  means  and  standard 
deviations  of  the  predictor  measures  overall  CV  sample.  Tables  2-4 
show  the  mean  and  standard  deviations  for  each  of  the  attribute 
measures  in  the  samples  for  each  of  the  three  Phase  I  MOS.  The 
estimated  standard  deviations  for  the  Youth  Population  are  also 
shown.  (The  means  for  the  Youth  Population  are  not  used  in  the 
following  analyses  and  so  were  not  estimated.) 

Method 

Once  the  covariances  of  the  predictor  and  criterion  measures 
were  estimated  for  each  job,  validities  for  any  given  composite  of 
the  predictors  can  be  estimated  through  relatively  direct  matrix 
manipulations.  There  are  two  steps  In  forming  a  predictor  composite 
score.  First,  the  attribute  scores  are  weighted  (by  estimated 
validities)  and  summed  to  form  a  predicted  score  for  each  job 
component.  Second,  these  predicted  job  component  scores  are  then 
weighted  (according  to  job  description  ratings)  and  summed  to  form 
the  predicted  total  job  performance  score. 


84 


Table  1 

Predictor  Means  and  Standard  Deviations  for  Each  Sample 


VARIABLE 

N 

ALL  MOS 

MEAN  STD  DEV 

1980 

POPULATION 

STD  DEV 

ASVAB  Subtests 

A1AS80GS:  General  Science 

7045 

51.40 

8.13 

10.00 

A1AS80AR:  Arithmetic  Reasoning 

7045 

52.87 

7.28 

10.00 

A1AS80VE:  Verbal  (WK  i  PC) 

7045 

50.96 

6.44 

10.00 

A1AS80NO:  Numeric  Operations 

7045 

52.71 

6.38 

10.00 

A1AS80CS:  Coding  Speed 

7045 

51.28 

6.68 

10.00 

A1AS80AS:  Auto/Shop  Information 

7045 

54.14 

8.53 

10.00 

A1AS80MK:  Mathematics  Knowledge 

7045 

50.98 

7.39 

10.00 

A1AS80MC:  Mechanical  Comprehension 

7045 

53.11 

8.17 

10.00 

A1AS80EI:  electronics  Information 

7045 

52.14 

7.55 

10.00 

Synthetic  Validity  Attribute  Measures 

SVATTROl:  Verbal  Ability 

7045 

102.37 

13.51 

18.97 

SVATTR02:  Reasoning 

7045 

102.44 

16.46 

19.27 

SVATTR03:  Number  Ability 

7045 

100.00 

17.40 

25.35 

SVATTR04:  Spatial  Ability 

7045 

100.00 

17.43 

21.18 

SVATTR06;  Mental  Info.  Processing 

7045 

100.00 

23.59 

24.71 

SVATTR07:  Perceptual  Speed  &  Acc. 

7045 

100.00 

17.64 

20.43 

SVATTR08:  Memory 

7045 

50.00 

14.22 

14.95 

SVATTR09:  Mechanical  Comprehension 

7045 

133.33 

17.63 

22.85 

SVATTRIO:  Eye-Limb  Coordination 

7045 

0 

14.01 

14.78 

SVATTRll:  Precision 

0 

18.84 

20.39 

SVATTR12:  Movement  Judgment 

7045 

6.62 

9.00 

9.38 

SVATTR13:  Hand  &  Finger  Dexterity 

7045 

16.73 

7.76 

7.86 

SVATTR17:  Involvement  in  Athletics 

7045 

13.90 

3.06 

3.07 

SVATTR18:  Work  Orientation 

7045 

150.00 

26.12 

26.76 

SVATTR20:  Cooperation/Stability 

7045 

150.00 

26.40 

26.94 

SVATTR21:  Energy 

7045 

48.43 

5.99 

6.09 

SVATTR22:  Conscientiousness 

7045 

102.48 

16.52 

16.66 

SVATTR23:  Dominance/Confidence 

7045 

100.00 

18.12 

18.92 

SVATTR24:  Interest  in  Using  Tools 

7045 

200.00 

32.93 

34.79 

SVATTR25:  Interest  in  Rugged  Act. 

7045 

150.00 

26.01 

26.46 

SVATTR26:  Interest  in  Protective  Serv. 

7045 

100.00 

17.03 

17.20 

SVATTR27:  Interest  in  Technical  Act. 

7045 

150.00 

23.55 

23.57 

SVATTR28:  Interest  in  Science 

7045 

200.00 

29.23 

29.51 

SVATTR29;  Interest  in  Leadership 

7045 

40.07 

8.45 

8.59 

SVATTR30:  Interest  in  Artistic  Act. 

7045 

14.13 

4.10 

4.16 

SVATTR31:  Interest  in  Efficiency  &  Org. 

7045 

200.00 

29.95 

30.71 
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Table  2 


Predictor  Means  and  Standard  Deviations  for  Each  Sample 


VARIABLE 


IIB:  Infantryman 


N  MEAN  STD  DEV 


Synthetic  Validation  Attribute  Measures 


SVATTROl:  Verbal  Ability 

491 

104.80 

13.34 

SVATTR02:  Reasoning 

491 

103.74 

16.95 

SVATTR03:  Number  Ability 

491 

100.00 

16.89 

SVATTR04:  Spatial  Ability 

491 

100.00 

17.69 

SVATTR06:  Mental  Info.  Processing 

491 

100.00 

23.75 

SVATTR07;  Perceptual  Speed  &  Acc. 

491 

100.00 

17.30 

SVATTR08;  Memory 

491 

50.00 

14.21 

SVATTR09:  Mechanical  Comprehension 

491 

137.41 

15.48 

SVATTRIO:  Eye-Limb  Coordination 

491 

0.00 

13.80 

SVATTRll:  Precision 

491 

0.00 

18.80 

SVATTR12:  Movement  Judgment 

491 

8.26 

7.92 

SVATTR13;  Hand  &  Finger  Dexterity 

491 

17.75 

7.34 

SVATTR17:  Involvement  in  Athletics 

491 

14.51 

2.96 

SVATTR18:  Work  Orientation 

491 

150.00 

26.07 

SVATTR20:  Cooperation/Stability 

491 

150.00 

25.95 

SVATTR21 :  Energy 

491 

47.72 

6.09 

SVATTR22:  Conscientiousness 

491 

98.18 

17.49 

SVATTR23;  Dominance/Confidence 

491 

100.00 

18.21 

SVATTR24:  Interest  in  Using  Tools 

491 

200.00 

32.08 

SVATTR25:  Interest  in  Rugged  Act. 

491 

150.00 

25.90 

SVATTR26:  Interest  in  Protective  Serv. 

491 

100.00 

17.35 

SVATTR27:  Interest  in  Technical  Act. 

491 

150.00 

23.20 

SVATTR28;  Interest  in  Science 

491 

200.00 

30.23 

SVATTR29:  Interest  in  Leadership 

491 

40.38 

8.90 

SVATTR30:  Interest  in  Artistic  Act. 

491 

14.21 

4.26 

SVATTR31:  Interest  in  Efficiency  &  Org. 

491 

200.00 

31.11 

Performance  Criterion  Measure 


M3RAWCTP:  Core  Technical  Prof. 


491  102.89  16.08 


Table  3 

Predictor  Means  and  Standard  Deviations  for  Each  Sample 


63B:  Vehicle  Mechanic 


VARIABLE 

N 

MEAN 

STD  DEV 

Synthetic 

Validation  Attribute  Measures 

SVATTROl 

Verbal  Ability 

478 

100.21 

12.84 

SVATTR02 

Reasoning 

478 

103.78 

16.10 

SVATTR03 

Number  Ability 

478 

100.00 

17.21 

SVATTR04 

Spatial  .  oility 

478 

100.00 

17.50 

SVATTR06 

Mental  Info.  Processing 

478 

100.00 

24.21 

SVATTR07 

Perceptual  Speed  &  Acc. 

478 

100.00 

17.23 

SVATTR08 

Memory 

478 

50.00 

13.80 

SVATTR09 

Mechanical  Comprehension 

478 

138.99 

16.17 

SVATTRIO 

Eye-Limb  Coordination 

478 

0.00 

14.19 

SVATTRll 

Precision 

478 

0.00 

18.83 

SVATTR12 

Movement  Judgment 

478 

6.96 

8.41 

SVATTR13 

Hand  &  Finger  Dexterity 

478 

15.72 

7.94 

SV\TTR17 

Involvement  in  Athletics 

478 

13.63 

3.09 

SVATTR18 

Worx  Orientation 

478 

150.00 

26.36 

SVATTR20 

Cooperation/Stability 

478 

150.00 

26.73 

SVATTR21 

Energy 

478 

47.62 

6.26 

SVATTR22 

Conscientiousness 

478 

99.76 

17.31 

SVATTR23 

Oominance/Conf idence 

478 

100.00 

17.80 

SVATTR24 

Interest  in  Using  Tools 

478 

200.00 

31.49 

SVATTR25 

Interest  in  Rugged  Act. 

478 

150.00 

25.78 

SVATTR26 

Interest  in  Protective  Serv. 

478 

100.00 

17.16 

SVATTR27 

Interest  in  Technical  Act. 

478 

150.00 

24.23 

SVATTR28 

Interest  in  Science 

478 

200.00 

30.75 

SVATTR29 

Interest  in  Leadership 

478 

37.53 

8.29 

SVATTR30 

Interest  in  Artistic  Act. 

478 

12.55 

3.96 

SVATTR31 

Interest  in  Efficiency  &  Org. 

478 

200.00 

30.66 

Performance  Criterion  Measure 


M3RAWCTP:  Core  Technical  Prof. 


478  102.62  15.23 


Table  4 


Predictor  Means  and  Standard  Deviations  for  Each  Sample 


VARIABLE 

71L:  Administrative  Specialists 

N 

MEAN 

STD  DEV 

Synthetic 

Validation  Attribute  Measures 

SVATTROl 

Verbal  Ability 

427 

100.77 

13.06 

SVATTR02 

Reasoning 

427 

100.93 

16.78 

SVATTR03 

Number  Ability 

427 

100.00 

17.25 

SVATTR04 

Spatial  Ability 

427 

100.00 

17.09 

SVATTR06 

Mental  Info.  Processing 

427 

100.00 

24.99 

SVATTR07 

Perceptual  Speed  &  Acc. 

427 

100.00 

18.45 

SVATTR08 

Memory 

427 

50.00 

14.02 

SVATTR09 

Mechanical  Comprehension 

427 

120.05 

16.33 

SVATTRIO 

Eye-Limb  Coordination 

427 

0.00 

13.71 

SVATTRll 

Precision 

427 

0.00 

18.66 

SVATTR12 

Movement  Judgment 

427 

2.22 

10.82 

SVATTR13 

Hand  &  Finger  Dexterity 

427 

15.05 

7.92 

SVATTR17 

Involvement  in  Athletics 

427 

13.01 

3.46 

SVATTR18 

Work  Orientation 

427 

150.00 

25.37 

SVATTR20 

Cooperation/Stability 

427 

150.00 

26.54 

SVATTR21 

Energy 

427 

49.29 

6.12 

SVATTR22 

Conscientiousness 

427 

108.80 

14.21 

SVATTR23 

Dominance/Confidence 

427 

100.00 

18.39 

SVATTR?4 

Interest  in  Using  Tools 

427 

200.00 

34.53 

SVATTR25 

Interest  in  Rugged  Act. 

427 

150.00 

26.18 

SVATTR26 

Interest  in  Protective  Serv. 

427 

100.00 

17.10 

SVATTR27 

Interest  in  Technical  Act. 

427 

150.00 

23.15 

SVATTR28 

Interest  in  Science 

427 

200.00 

27.63 

SVATTR29 

Interest  in  Leadership 

427 

41.94 

8.26 

SVATTR30 

Interest  in  Artistic  Act. 

427 

15.84 

4.01 

SVATTR31 

Interest  in  Efficiency  &  Org, 

427 

200.00 

28.82 

Performance  Criterion  Measure 

M3RAWCTP 

:  Core  Technical  Prof, 

427 

101.16 

17.36 

In  addition  to  the  synthetically  produced  predictor  composites, 
we  developed  "empirical"  prediction  equations  using  least-squares 
regression.  When  the  same  empirical  data  were  used  to  estimate  the 
validity  of  the  empirical  composites,  an  adjustment  was  applied  to 
give  unbiased  estimates  of  cross-validation  validities  for  these 
composites  (Darlington,  1968,  and  Herzberg,  1969;  both  gave  the  same 
results).  No  adjustments  were  made  when  we  estimated  the  validity  of 
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the  empirical  equation  developed  for  one  job  for  predicting  performance 
in  a  different  job,  since  the  criterion  data  for  the  other  job  were 
not  used  in  the  development  of  the  empirical  weights. 

In  addition  to  the  three  empirical  composites,  validities  were 
estimated  for  each  of  the  three  synthetic  composites  derived  from 
each  of  the  three  job  component  models.  In  each  case,  we  estimated 
validities  for  each  of  the  composites  for  predicting  each  of  the 
three  jobs.  Differences  between  the  validity  of  each  composite  for 
the  intended  job  and  the  average  of  the  validities  for  each  of  the 
other  two  jobs  were  used  as  an  indicator  of  discriminant  validity. 

Initial  results  (described  below)  showed  high  absolute  validities, 
but  relatively  low  levels  of  discriminant  validity  for  the  synthetic 
composites.  Further  analyses  of  alternative  methods  for  forming  the 
attribute  weights  were  conducted. 

Two  alternative  methods  for  forming  the  attribute-by-component 
weights  (for  predicting  performance  at  the  component  level)  and  one 
alternative  for  forming  the  component-by- job  weights  (for  weighting 
the  individual  prediction  equations  to  form  an  overall  equation)  were 
explored.  The  initial  method  for  developing  prediction  equations  for 
each  job  component  (attribute-by-component  weights)  used  attribute 
weights  that  were  proportional  to  the  attribute-by-component  validities 
estimated  by  the  expert  psychologists.  The  first  alternative  to 
these  validity  weights  was  to  compute  "regression"  weights  that  t''''k 
the  correlations  among  the  predictor  measures  into  account.  (In 
matrix  terms,  the  regression  weights  are  given  by  the  product  of  the 
validity  vector  with  the  inverse  of  the  matrix  of  predictor 
correlations.)  The  second  alternative  was  to  use  unit  weights.  In 
this  second  alternative,  all  attributes  with  mean  validity  ratings 
less  than  3.5  were  given  a  weight  of  0  and  all  remaining  attributes 
were  given  a  weight  of  1  for  combining  standardized  predictor  scores 
into  job  component  performance  estimates. 

The  initial  method  used  to  form  composite-by-job  weights  was  to 
use  the  mean  of  the  Subject  Matter  Expert  (SME)  ratings  of  the 
importance  of  the  component  for  core  technical  proficiency  in  the 
job.  These  ratings  were  on  a  scale  from  0  to  4.74  that  comprised  a 
somewhat  arbitrary  metric.  The  alternative  used  to  compute  "adjusted" 
weights  was  to  use  a  value  of  0  where  the  mean  importance  rating  was 
less  than  2.5  and  to  otherwise  use  the  square  of  the  mean  importance 
ratings  so  as  to  increase  the  relative  weight  given  to  the  more 
important  components.  The  objective  in  analyzing  composites  derived 
from  these  alternative  component  weights  was  not  to  find  a  method 
with  a  better  rationale  so  much  as  to  explore  the  sensitivity  of  the 
final  results  to  differences  in  the  scaling  of  these  weights. 


(' 
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Attribute 


1  2  3 

4 

6 

7 

8  9 

10 

11 

12 

13 

17  18 

20 

21 

22 

24 

25 

27 

28 

29 

30 

11B: 

Infantryman 

EM11B 

306  153  113 

117 

-13 

169 

91  187 

8 

35 

-23 

45 

28  59-1U 

-33  138 

47 

22  157 

-59 

81 

-79 

28 

-61 

-40 

TK11B 

130  129  91 

106  101 

97 

108  84 

78 

73 

67 

75 

57  103 

73 

83 

90 

74 

69 

67 

61 

62 

43 

60 

20 

66 

AC11B 

116  110  79 

91 

96 

90 

100  72 

82 

78 

77 

68 

86  107 

85 

97 

95 

83 

68 

78 

60 

59 

54 

76 

43 

70 

AT11B 

99  98  87 

98 

106 

90 

86  116 

96 

92 

93 

84 

88  93 

67 

91 

79 

71 

88  113 

75 

48 

13 

63 

-22 

46 

VT11B 

116  110  79 

91 

96 

90 

100  72 

82 

78 

77 

68 

86  107 

85 

97 

95 

83 

68 

78 

60 

59 

54 

76 

43 

70 

VA11B 

95  90  67 

90 

88 

83 

81  63 

82 

84 

84 

65 

119  121 

93  119  102 

84 

67  106 

70 

53 

45 

75 

41 

67 

RTIIB 

857  277-326 

4  235 

-23 

174-371 

162 

-64 

116  168  302  154 

-8 

-48  285 

-59  382 

-40 

117 

-5-237 

-0 

-62  449 

RA11B 

671  132-280 

153  223 

-0 

100-371 

126 

-5 

133 

142  403  182 

-9 

43  289-164  189 

172 

109 

-41- 

253 

3 

-36  453 

UT11B 

217  165  39 

64 

37 

68 

134  27  112 

81 

69 

42 

75  95 

103 

118 

86 

103 

33 

48 

0 

12 

4 

103 

0 

0 

UA11B 

K7  60  0 

66 

23 

78 

87  0 

98 

98 

112 

0 

149  145 

98  219 

141 

98 

0 

113 

0 

0 

0 

98 

0 

0 

63B: 

Vehicle  Mechanic 

EH63B 

27  198  -22 

-13 

78 

79 

15  608 

-30 

7 

26 

76 

-68  96 

63-132 

154 

-31 

407 

83- 

107-157 

29 

-4 

-79 

-16 

TK638 

133  128  95 

105 

97 

97  106  98 

79 

75 

62 

80 

51  104 

69 

81 

92 

68 

79 

57 

53 

71 

45 

55 

19 

74 

AC63B 

113  111  82 

93 

97 

91 

101  85 

89 

83 

76 

77 

75  103 

78 

90 

92 

76 

80 

71 

56 

66 

56 

69 

40 

69 

AT63B 

114  121  105 

115 

91 

81 

101  163 

91 

75 

68 

96 

41  84 

50 

58 

74 

48  135 

63 

32 

99 

45 

41 

6 

65 

VT63B 

113  111  82 

93 

97 

91 

101  85 

89 

83 

76 

77 

75  103 

78 

90 

92 

76 

80 

71 

56 

66 

56 

69 

40 

69 

VA63B 

88  97  72 

90 

92 

83 

81  107  116  105 

80 

97 

81  108 

72 

92 

92 

65 

110 

81 

57 

79 

54 

54 

33 

64 

RT63B 

744  257-267 

-47  241 

-15 

178-207  199 

-70 

91 

208  263  200 

-32 

-73  285 

-68  456-150 

117 

57-241 

-22 

-47  437 

RA63S 

377  134- 160- 

■128  230 

•6 

99  175  310 

-20 

42  271 

257  320 

-75 

-55  283- 

176  457-218 

98 

ia-252 

-50 

3  386 

UT63B 

188  166  70 

67 

33 

53 

124  77  149  103 

56 

90 

49  76 

82 

92 

75 

82 

88 

35 

0 

33 

3 

82 

0 

0 

(JA63B 

73  115  81 

42 

0 

0 

34  170  261 

195 

40 

171 

51  79 

28  106  101 

28  187 

51 

0 

81 

0 

28 

0 

0 

71L:  Administrative  Clerk 

EM71L 

206  444  361 

114 

21 

-52 

96-161 

-94 

34 

-2 

23 

-63  111 

64 

-56  154- 

131- 

■180 

-23 

22 

15 

106 

48 

-41 

38 

TK71L 

160  136  101 

88 

99 

102 

111  73 

68 

64 

53 

72 

46  110 

81 

85 

97 

75 

55 

36 

49 

60 

47 

65 

30 

87 

AC71L 

144  123  89 

81 

102 

93 

119  65 

67 

64 

56 

69 

63  108 

90 

91 

102 

86 

54 

52 

55 

55 

58 

81 

46 

81 

AT71L 

161  126  113 

68 

118 

151 

124  56 

80 

82 

36  132 

25  106 

87 

85 

105 

53 

35 

5 

15 

40 

19 

38 

28 

122 

VT71L 

144  123  89 

81 

102 

93 

119  65 

67 

64 

.  56 

69 

63  108 

90 

91 

102 

86 

54 

52 

55 

55 

58 

81 

46 

81 

VA71L 

168  129  89 

76 

109 

100 

139  60 

64 

60 

46 

77 

49  110 

87 

81 

116 

72 

45 

35 

53 

52 

60 

66 

46 

100 

RT71L 

1125  402-357-172  208 

-43  223-510 

109 

-53 

35 

185  206  137 

44 

•101 

262 

-8  445 

■139 

131 

-18-222 

-9- 

•121 

452 

RA71L 

1283  387-451 

-220 

184 

-34  245-554 

87 

-18 

2  201 

159  233 

44-183  269 

-52  400-169 

138 

- 18-169- 

■123- 

•150  498 

UT71L 

305  211  55 

34 

46 

48 

177  12 

41 

32 

24 

TO 

26  83  116 

83 

104 

116 

12 

17 

0 

6 

2  116 

0 

0 

UA71. 

525  22S  0 

0 

0 

0  216  0 

0 

0 

0 

117 

0  54 

54 

54  263 

54 

0 

0 

0 

0 

0 

54 

0 

0 

COMPOSITE  COPE; 

EH  -  Eitpirical  composite 

VT  ■  Validity  attribute  weights;  total  component  weights 
VA  -  Validity  attribute  weights;  adjusted  conp.  weights 
AT  •  Regression  attribute  weights;  total  component  weights 
RA  -  Regression  attribute  weights;  adjusted  comp,  weights 


UT  -  Unit  attribute  weights;  total  component  weights 
UA  *  Unit  attribute  weights;  adjusted  component  weights 
TK  •  Task  category  weights 
AC  -  Activity  weights 
AT  •  Attribute  wei^ts 
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Table  5  shows  attribute  coefficients  for  each  of  the  different 
predictor  composites  investigated.  These  coefficients  have  been 
scaled  so  as  to  produce  scores  of  unit  variance  when  applied  to 
standardized  predictor  scores  in  the  1980  Youth  Population.  One 
result  that  is  evident  in  Table  2  is  that  the  initial  and  "unit-weight" 
synthetic  composites  do  not  include  negative  weights.  The  empirical 
composites  do  include  some  negative  weights,  although  generally 
relatively  small  ones.  The  "regression-weight"  synthetic  composites, 
on  the  other  hand,  include  a  greater  number  of  negative  weights  with 
somewhat  larger  values. 

Results 


Table  6  shows  the  validity  estimates  for  the  synthetic  and 
empirical  predictor  composites.  For  the  Attribute  Model,  both 
psychologist  and  soldier-derived  composites  are  shown. 

Table  7  shows  the  mean  across  the  three  jobs  of  the  diagonal 
(appropriate)  validities  for  each  set  of  predictor  composites.  This 
value  represent  the  mean  absolute  validity.  Table  7  also  shows  (in 
parentheses)  the  difference  between  each  of  these  means  and  the 
corresponding  mean  of  the  off-diagonal  (inappropriate)  validities. 

This  difference  statistic  can  be  interpreted  as  a  measure  of  the  mean 
di scriminant  validity. 

The  empirical  validities  are  quite  high  (a  mean  validity  of  .673 
across  the  three  MOS)  and  show  moderate  discrimination  (a  mean  of 
.173  across  the  three  MOS).  The  main  result  indicates  that  the 
different  approaches  to  weighting  the  attributes  led  to  significant 
differences  in  absolute  validities.  The  regression  weights  led  to 
low  validities  (.34  to  .40).  The  validity  weights  led  to  relatively 
high  validities  (.52  to  .58).  In  all  cases  except  the  psychologist- 
based  attribute  weights,  the  unit  weights  led  to  even  higher  validities 
(.53  to  .63).  In  all  but  one  case,  the  use  of  unit  weights  led  to 
slightly  higher  discriminant  validities. 

The  adjustments  to  the  component  weights  had  mixed,  but  generally 
small,  effects  on  the  absolute  validities.  The  adjusted  component 
weights  did  lead  to  consistently  higher  discriminant  validities. 

Overall,  the  highest  validities  were  derived  from  the  Task 
Model.  The  highest  mean  validity  was  .63,  very  close  to  the  mean  for 
the  empirical  composite  of  .67.  Ignoring  the  regression  attribute 
weights,  the  greatest  discriminant  validity  was  obtained  from  the 
Activity  Model.  The  maximum  difference  here  was  .14,  compared  to  a 
difference  for  the  empirical  composite  of  .17. 
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Table  6 


Validities  for  Each  Prediction  Eouation  Against  Infantry  (llBl. 
Vehicle  Mechanic  (63BK  and  Administrative  Clerk  (71L1 


Prediction 

Weighting 

Method 

Predict 

Equation 

MOS 

Task  Model 
Criterion 
IIB  63B  71L 

Activity  Model 
Criterion 

IIB  63B  71L 

Attribute  Model 
Weighting  Criterion 
Method  IIB  63B  71L 

IIB 

63 

51 

50 

60 

48 

47 

63 

53 

45 

V  X  T 

63B 

64 

51 

50 

61 

49 

48 

Vx 

66 

56 

48 

71L 

63 

49 

52 

60 

46 

50 

Psych 

64 

49 

54 

IIB 

63 

50 

49 

57 

46 

44 

57 

45 

44 

V  X  A 

63B 

65 

54 

49 

60 

51 

45 

Vx 

60 

50 

45 

71L 

64 

49 

54 

61 

46 

52 

Sold 

59 

45 

49 

IIB 

43 

36 

33 

40 

33 

30 

43 

43 

23 

R  X  T 

63B 

42 

38 

32 

41 

38 

30 

Rx 

41 

48 

24 

71L 

37 

29 

33 

37 

28 

33 

Psych 

30 

22 

28 

61B 

42 

34 

32 

36 

29 

25 

35 

30 

24 

R  X  A 

63B 

41 

45 

25 

38 

42 

21 

Rx 

36 

42 

20 

71L 

30 

21 

28 

32 

24 

31 

Sold 

29 

23 

26 

IIB 

71 

56 

55 

66 

50 

50 

59 

49 

44 

U  X  T 

63B 

71 

58 

55 

67 

54 

50 

Ux 

59 

48 

46 

71L 

69 

51 

58 

68 

49 

57 

Psych 

62 

50 

49 

IIB 

70 

56 

53 

54 

41 

39 

61 

51 

45 

U  X  A 

63B 

72 

62 

51 

61 

55 

39 

Ux 

62 

50 

49 

71L 

68 

49 

56 

67 

47 

59 

Sold 

55 

40 

48 

Empirical  Prediction 
IIB  63B 

Empirical  IIB  72  59 

63B  61  68 

71L  59  35 

V  »  Validity  weights 
R  =  Regression  weights 
U  =  Unit  weights 


m 

53 

33 

62 

T  «  Task  category  weights 
A  -  Job  activity  weight 
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Table  7 


Mean  Validities:  Absolute  Validities  and  Discriminant  Validities  (Differences 
from  Off-Diagonal  Validities! 


Attribute 

Weights 

Task  Model 

Comoonent  Wts 
Total  Adj 

Activity  Model 

Comoonent  Wts 
Total  Adj 

Attribute  Model 

Psych  Soldier 

Val idity 

.553 

.570 

.530  .533 

.577 

.520 

Wts 

(.008) 

(.027) 

(.013)(.031) 

(.035) 

(.023) 

Regr 

.380 

.383 

.370  .363 

.397 

.343 

Wts 

(.032) 

(.078) 

(.038)(.081) 

(.164) 

(.073) 

Unit 

620 

.627 

.590  .560 

.520 

.530 

Wts 

(.025) 

(.045) 

(.033)(.138) 

(.003) 

(.027) 

Note:  The  mean  validity  for  the  empirically  derived  equation 
(adjusted  for  shrinkage)  is  .673.  This  is  .173  greater 
than  the  mean  off-diagonal  validity  for  these  equations. 


Discussion 

The  results  of  the  comparative  analyses  are  generally  supportive 
of  the  synthetic  validation  approach.  They  indicate,  however,  that 
further  work  is  needed  to  evaluate  the  alternative  approaches  for 
forming  the  composite  weights  from  the  information  gathered.  The 
exploratory  nature  of  the  analysis  of  alternative  adjustments  together 
with  the  very  small  number  of  jobs  studied  strongly  suggest  the  need 
for  cross-validation  through  confirmatory  analyses  on  additional 
samples.  Fortunately,  such  efforts  are  planned  as  part  of  Phase  II 
and  Phase  III  of  the  current  project. 

The  initial  results  do  not  show  a  strong  preference  for  any  one 
of  the  job  component  models.  The  effects  of  alternative  weight 
estimation  procedures  were  much  greater  than  any  differences  between 
the  different  models.  The  tendency  toward  relatively  uniform  weighting 
of  the  different  attributes  suggests  the  need  to  consider  both 
attribute  and  job  descriptor  models  that  are  more  parsimonious  so 
that  the  contrast  in  the  weights  will  be  more  significant.  In 
addition,  the  use  of  umt  weight  models  reduces  the  number  of  different 
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tests  that  are  used  in  predicting  performance  for  each  job,  resulting 
in  potential  reductions  in  required  testing  time. 
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CHAPTER  7: 


Exploring  the  Relationships  among  Rater  Characteristics 
and  Fidelity  of  Job  Description  Judgments 

Philip  L.  Szenas 
Jeffrey  J.  McHenry 
American  Institutes  for  Research 


SUMMARY 

The  purpose  of  this  research  was  to  identify  characteristics  of 
individual  raters  who  provide  job  description  judgments  that  are 
consistent  with  the  consensus  job  description  ratings  provided  by  all 
raters  within  a  group  (i.e.,  high  rating  fidelity).  A  tentative 
model  of  the  process  variables  involved  in  job  description  was 
developed.  Three  individual  difference  characteristics  were  identified 
and  measured:  job  experience,  job  knowledge,  and  general  aptitude. 
Eighty  non-commissioned  officers  (NCOs)  from  three  jobs  served  as 
judges.  Judges  completed  job  description  questionnaires  representing 
three  distinct  approaches  to  describing  jobs:  task-based,  activity- 
based,  and  attribute-based  approaches.  Relationships  among  the  three 
individual  difference  constructs  and  rating  fidelity  were  examined 
via  path  analysis.  Results  suggest  that  the  data  fit  the  hypothesized 
model  very  well.  The  model  also  did  a  good  job  of  predicting  which 
judges  would  provide  high-fidelity  judgments.  Implications  for 
selecting  job  description  judges  and  goals  of  future  research  are 
provided. 


CHAPTER  7: 


Exploring  the  Relationship  among  Judge  Characteristics 
and  Fidelity  of  Job  Description  Judgments 


The  purpose  of  this  research  was  to  identify  the  characteristics 
of  raters  who  provide  job  description  judgments  that  are  consistent 
with  job  description  judgments  provided  by  other  raters  (i.e.,  rating 
fidelity).  There  are  both  theoretical  and  practical  reasons  why  this 
issue  is  important.  First,  by  understanding  the  underlying 
knowledge,  skills,  abilities,  and  other  personal  characteristics 
(KSAOs)  that  are  involved  in  job  description  judgments,  we  may  begin 
to  develop  a  theoretical  map  of  the  process  variables  involved. 
Second,  most  organizations  generate  and  use  job  descriptions  for  a 
variety  of  human  resource  functions.  This  research  may  help  suggest 
who  is  qualified  to  provide  the  judgments  that  are  required  to 
produce  these  job  descriptions. 

There  has  been  very  little  research  on  the  individual  difference 
correlates  of  job  description  judgments.  However,  there  have  been  a 
number  of  studies  that  have  investigated  the  effects  of  individual 
difference  variables  on  performance  ratings  (Bernardin,  Cardy  & 
Carlyle,  1982;  Zedeck  &  Kafry,  1977;  Cascio  &  Valenzi,  1978;  Kirchner 
&  Reisberg,  1962).  In  a  summary  of  this  literature,  Cascio  (1987) 
reported  that:  (a)  interests,  social  insight,  intelligence,  and 
cognitive  complexity  show  no  consistent  relationship  with  rating 
fidelity  and  accuracy;  (b)  the  education  level  and  job  experience  of 
the  judge  have  small,  positive  correlations  with  fidelity  and 
accuracy;  and  (c)  the  job  performance  of  the  judge  has  a  substantial 
positive  correlation  with  rating  fidelity  and  accuracy. 

To  provide  accurate,  reliable  performance  ratings,  a  judge  must 
be  able  to  observe  and  evaluate  behavior.  The  requirements  of  job 
description  judgments  are  somewhat  different.  No  evaluation  of  the 
effectiveness  of  behavior  is  required.  Instead,  the  judge  must  be 
able  to  observe  and  describe  behavior  accurately.  If  the  job  is 
described  in  terms  of  tasks,  the  judge  may  be  required  to  infer  the 
purpose  of  the  behavior  (i.e.,  what  the  incumbent  is  attempting  to 
accomplish  on  the  job).  If  the  job  is  described  in  terms  of  ability 
or  skill  requirements,  the  judge  may  be  required  to  infer  these  from 
the  behavior  performed  on  the  job. 

In  spite  of  these  differences,  we  believe  that  the  findings  from 
the  performance  rating  literature  may  be  relevant  for  the  present 
research.  Specifically,  we  hypothesized  that  three  individual 
difference  constructs  contribute  to  the  fidelity  of  job  description 
judgments:  job  experience,  job  knowledge,  and  general  aptitude.  Job 
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experience  and  job  knowledge  are  salient  for  the  task  of  defining  the 
job.  In  order  for  a  judge  to  accurately  define  a  job,  he  or  she  must 
have  the  experience  and  knowledge  to  identify  the  job  elements  that 
are  and  are  not  part  of  the  job  and  be  able  to  determine  the 
criticality  of  each  job  element.  General  aptitude  helps  the  judge 
understand  the  judgment  task.  General  aptitude  also  contributes  to 
abstract  thinking  skills,  enabling  the  judge  to  link  elements  from 
the  job  description  questionnaire  to  the  behaviors  performed  on  the 
job. 


Figure  1  represents  a  theoretical  model  of  how  the  specific 
judge  characteristics  (described  below)  used  in  our  analyses  affect 
the  quality  of  the  job  description  judgments.  As  the  figure  shows, 
job  experience  and  general  aptitude  combine  to  influence  both 
knowledge  of  the  judges'  own  jobs  and  the  jobs  they  are  rating.  The 
effect  of  job  tenure  on  rating  fidelity  is:  direct  (path  11),  through 
job  knowledge  (paths  7-13,  and  9-14),  and  through  on-the-job 
experience  and  job  knowledge  (paths  1-3-13,  and  1-4-14).  The  effect 
of  general  aptitude  on  rating  fidelity  mirrors  the  effects  of  job 
tenure  in  that:  it  has  a  direct  effect  (path  12),  acts  through  job 
knowledge  directly  (paths  10-13,  and  8-14),  and  acts  through 
education  and  job  knowledge  (paths  2-5-15,  and  2-6-14). 

Although  the  paths  outlined  are  consistent  across  all  three 
judgment  job  description  questionnaires,  it  is  quite  possible  that 
the  levels  of  the  individual  difference  constructs  necessary  for  the 
judges  to  provide  good  judgments  will  differ  from  questionnaire  to 
questionnaire.  Clearly,  less  abstraction  is  required  of  the  judges 
for  the  identification  of  tasks  that  are  important  for  jobs,  while 
much  greater  abstraction  is  required  for  the  leap  in  linking 
attributes  and  jobs.  Therefore,  it  is  possible  that  more  general 
aptitude  is  required  for  judges  to  provide  high-fidelity  attribute- 
based  job  description  judgments  than  is  required  for  high-fidelity 
task-based  judgments.  Conversely,  a  higher  level  of  specific  job 
knowledge  may  be  required  to  make  the  task-based  judgments  than  is 
required  for  the  attribute-based  judgments. 


Mgthgd 


Judges 

Eighty  non-commissioned  officers  (NCOs),  representing  three  Army 
enlisted  specialties  (Infantryman,  Vehicle  Mechanic,  and 
Administrative  Specialist),  served  as  judges.  The  judges  were  chosen 
to  include  representative  samples  of  blacks,  Hispanics,  whites, 
females,  and  males  for  each  specialty.  The  judges  averaged  3.5  years 
in  the  Army  and  2.8  years  in  their  specialty. 
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Job  Description  Questionnaires 


Three  job  description  questionnaires  were  developed.  The  purpose 
of  these  questionnaires  was  to  provide  a  description  of  the 
responsibilities  and  requirements  of  soldiers  with  18  -  24  months  job 
experience  (i.e.,  first-tour  soldiers)  in  any  of  the  265  Army  enlisted 
specialties  open  to  new  recruits. 

The  first  questionnaire  was  a  task-based  instrument  consisting 
of  96  task  categories.  The  task-based  approach  to  job  description 
has  as  its  focus  the  inherent  characteristics  of  tasks  (i.e.,  the 
work  itself).  The  major  assumptions  of  this  approach  are  that  the 
job  tasks  "impose  certain  conditions  on  an  individual  performing 
them,  such  as  the  goals  to  be  achieved,  procedures  to  be  followed, 
characteristics  of  the  responses  elicited  from  the  task  performer, 
and  activity  content"  (Crafts,  Szenas,  Chia,  &  Pulakos,  1988,  p.  21). 
An  example  task  category  is  "Send  and  receive  radio  messaqes--use 
standardized  radio  codes  and  procedures  to  transmit  and  receive 
messages  and  other  information." 

Judges  were  asked  to  consider  the  job  of  a  soldier  with  18  -  24 
months  experience  in  their  specialty  (i.e.,  the  first-tour  soldiers 
that  these  NCOs  supervised).  Judges  then  were  instructed  to  provide 
three  importance  ratings  for  each  task  category: 

•  the  importance  of  the  task  category  for  the  job- 
specific/technical  component  of  their  first-tour  soldiers' 
jobs; 

•  the  importance  of  the  task  category  for  the  general 
soldiering  component  of  their  first-tour  soldiers'  jobs 
(i.e.,  for  the  responsibilities  such  as  navigating  and 
first  aid  that  are  a  part  of  every  soldier's  job);  and 

•  the  overall  importance  of  the  task  category  for  their 
first-tour  soldiers'  jobs. 

The  second  instrument  was  an  activity-based  questionnaire 
consisting  of  53  job  activities  or  behaviors.  The  activity-based 
approach  focuses  on  worker  behaviors.  An  example  activity  is 
"Monitor/interoret  verbal  messaqes--monitor.  identify  and/or  interpret 
verbal  messages  (oral  or  written,  obtained  from  radio,  teletype, 
computer  terminal,  correspondences,  etc.)."  Judges  made  job-specific, 
general  soldiering,  and  overall  importance  ratings  for  each  activity. 

The  third  questionnaire  consisted  of  31  attributes.  The 
attribute-based  approach  to  job  description  focuses  on  the  abilities, 
traits,  interests  and  other  individual  differences  that  contribute  to 
successful  job  performance.  An  example  attribute  is  "Verbal 
abil itv--this  is  the  ability  to  use  and  understand  spoken  and  written 
language  and  to  communicate  with  others.  It  involves  "catching  on" 
to  what's  happening  and  coming  up  with  and  understanding  words  and 
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ideas."  Judges  were  provided  with  a  brief  15-minute  explanation  of 
test  validity,  including  examples  of  tests  with  high-  and  low-validity 
for  particular  jobs.  They  then  were  asked  to  rate  the  validity  of 
each  attribute  for  predicting  job-specific  and  general  soldiering 
performance  of  first-tour  soldiers  in  their  specialty. 

Chia,  Owens-Kurtz,  Peterson,  and  Szenas  (1988)  and  Owens-Kurtz  & 
Peterson  (1989)  reported  mean  single  rater  reliabilities  across  the 
three  jobs  of  .53,  .35,  and  .31  for  the  judges  in  the  current  research, 
for  the  task-,  activity-,  and  attribute-based  questionnaires, 
respectively.  These  interrater  reliabilities  indicate  that  judges 
generally  agree  about  the  most  and  least  important  tasks  and  activities 
for  first-tour  soldiers  in  their  specialties.  The  results  also 
indicate  that  judges  agree  concerning  the  attributes  that  will  provide 
the  most  and  least  accurate  prediction  of  job-specific  and  general 
soldiering  performance  for  first-tour  soldiers. 

Assessment  of  Judge  Characteristics 

Three  additional  questionnaires  were  designed  to  provide  an 
assessment  of  judges'  job  experience  and  job  knowledge.  The  first 
was  a  background  information  questionnaire,  consisting  of  13 
demographic  items  such  as  gender,  race,  and  pay  grade.  The  second 
was  a  job  history  questionnaire.  The  questionnaire  contained  a  list 
of  30  critical  job  tasks  identified  for  each  of  the  three  specialties. 
Judges  were  asked  to  rate  the  number  of  times  they  had  performed  and 
supervised  each  task.  The  final  instrument  was  a  job  familiarity 
questionnaire  which  consisted  of  15  job  knowledge  test  items.  These 
items  assessed  the  NCOs'  knowledge  of  critical  first-tour  job  tasks 
for  their  specialty  and  were  sampled  from  approximately  100-120  job 
knowledge  items  that  were  developed  for  the  MOS  in  Project  A. 

These  three  questionnaires  and  additional  archival  data  collected 
from  Army  administrative  records  were  used  to  create  two  measures  of 
job  experience,  job  knowledge,  and  general  aptitude  for  each  judge. 

The  first  measure  of  job  experience  was  job  tenure,  defined  as  the 
number  of  years  experience  with  the  rated  job.  The  second  measure 
was  task  experience,  defined  as  the  number  of  tasks  on  the  job  history 
questionnaire  that  the  judge  had  performed  at  least  once. 

The  first  measure  of  job  knowledge  was  the  percent  of  correct 
items  on  the  job  familiarity  questionnaire.  The  second  job  knowledge 
measure  was  the  judge's  score  on  the  Army  Skill  Qualification  Test 
(SQT),  which  is  a  job  knowledge  test  that  the  Army  uses  to  assess  its 
enlisted  personnel.  The  chief  difference  between  the  SQT  and  our  job 
familiarity  questionnaire  was  that  the  SQT  included  items  about  NCO 
supervisory  responsibilities  and  technical  tasks  that  must  be  performed 
by  NCOS,  but  that  are  not  part  of  the  job  of  first-tour  soldiers. 

One  measure  of  general  aptitude  was  the  Armed  Forces  Qualification 
Test  (AFQT)  from  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB).  The  AFQT  composite  includes  both  verbal  and  mathematical 
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components.  The  second  general  aptitude  measure  was  educational 
attainment,  defined  as  the  number  of  years  of  formal  schooling. 

Rating  Fidelity 

The  fidelity  of  each  judge's  job  description  judgments  was 
defined  as  the  correlation  of  the  judge's  rating  profile  of  the 
questionnaire  elements  with  the  mean  profile  of  the  remaining  judges 
in  the  same  job.  For  example,  for  the  task-based  questionnaire,  the 
fidelity  coefficient  was  the  correlation  between  the  judge's  importance 
ratings  of  the  96  task  categories  and  the  mean  importance  ratings  of 
the  other  raters  in  the  judge's  specialty.  Thus,  three  fidelity 
coefficients  were  computed  for  each  judge--one  for  the  task-based 
questionnaire,  one  for  the  activity-based  questionnaire,  and  one  for 
the  attribute-based  questionnaire. 

Workshop  Procedures 

All  job  description  judgments  were  collected  in  a  single  four- 
hour  workshop.  Workshops  were  divided  into  two  sessions  with  a  15- 
minute  break  between  sessions.  One-half  of  the  judges  were  given  the 
task-based  and  activity-based  job  description  questionnaire  in  the 
first  session,  and  one-half  were  given  the  attribute-based  job 
description  questionnaire.  The  order  of  administration  of  the  task- 
and  activity-based  questionnaire  was  counterbalanced  within  each 
session.  The  judges  were  instructed  to  think  of  the  entire  range  of 
duty  assignments  that  make  up  the  job  of  incumbents  with  18  months  of 
service  (after  completing  basic  and  advanced  training).  They  were 
instructed  to  draw  from  their  experiences  with  the  job  as  it  is 
practiced  throughout  the  Army  and  not  just  in  their  current 
assignments. 

Statistical  Analyses 

Using  the  LISREL  VI  software  (Joreskog  i  Sorbom,  1983),  path 
analysis  was  used  to  test  how  well  the  model  in  Figure  1  fit  the 
observed  covariances  among  the  job  experience,  job  knowledge,  general 
aptitude  and  fidelity  measures.  Two  criteria  were  used  to  evaluate 
the  fit  of  the  model.  The  first  was  the  ratio  of  chi -squared  (X^)  to 
the  degrees  of  freedom  (df;  Schmitt,  1978).  The  second  criterion  was 
rho,  which  is  a  goodness  of  fit  index  developed  by  Tucker  and  Lewis 
(1973).  A  good  fit  is  found  when  X^/df  ratio  is  3:1  or  lower,  and 
rho  is  greater  than  .90.  In  addition,  the  multiple  correlations 
between  the  judge  characteristics  and  the  measures  of  rating  fidelity 
were  used  to  evaluate  how  well  the  model  predicted  judgment  quality. 

The  model  was  first  tested  for  goodness  of  fit  while  the  parameter 
estimates  for  the  paths  leading  directly  to  the  three  fidelity  criteria 
were  assumed  to  be  equal.  A  second  model  also  was  tested  in  which 
these  parameter  estimates  were  allowed  to  vary.  The  goodness-of-fit 
of  these  two  models  was  compared  to  determine  if  they  fit  the  observed 
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between  the  judge  characteristics  and  the  measures  of  rating  fidelity 
were  used  to  evaluate  how  well  the  model  predicted  judgment  quality. 

The  model  was  first  tested  for  goodness  of  fit  while  the 
parameter  estimates  for  the  paths  leading  directly  to  the  three 
fidelity  criteria  were  assumed  to  be  equal.  A  second  model  also  was 
tested  in  which  these  parameter  estimates  were  allowed  to  vary.  The 
goodness-of-fit  of  these  two  models  was  compared  to  determine  if  they 
fit  the  observed  correlations  significantly  better  when  the  paths  to 
the  three  fidelity  coefficients  were  allowed  to  vary. 


Results 


Table  1  contains  the  means  and  standard  deviations  of  the  six 
individual  difference  measures  and  the  three  fidelity  coefficients. 
The  correlations  among  these  measures  are  shown  in  Table  2.  The 
correlations  among  the  fidelity  coefficients  are  underlined. 


Table  1 

Descriptive  Statistics 


Variable 

Type  of  measure 

Mean* 

Std  Dev 

General  aptitude 

Percentile  Score 

47.90 

23.46 

Time  in  job 

Years 

2.83 

1.32 

Educational  level  attained 

Year 

t2.60 

1.15 

Experience  on  the  job 

Total  Items  Indorsed 

69.33 

30.01 

Knowledge  of  job  rated 

Proportion  Correct 

0.67 

0.13 

Knowledge  of  own  job 

Percentile  Score 

49.60 

32.37 

Fidelity  of  task-based 
judgments 

Correlation  X  100 

49.49 

10.72 

Fidelity  of  activity- 
based  judgments 

Correlation  X  100 

50.33 

10.16 

Fidelity  of  attribute- 
Based  judgments 

Correlation  X  100 

50.34 

9.94 

*N  *  80  judges 

I 

V  \ 
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Table  2 

Correlation  Matrix 


XI  X2  X3  X4  X5  X6 

Yl  Y2  Y3 

XI:  Job  tenure 

1.000 

X2:  General  Ability 

-.119  1.000 

X3:  Familiarity 
with  job  rating 

.113  .063  1.000 

X4:  Educational 
level  attained 

.050  .357*-. 077  1.000 

X5:  Job  Knowledge 
(of  job  rated) 

.183  .323*  .066  .283*1.000 

X6:  Job  Knowledge 
(of  own  job) 

-.121  .304*  .024  .298*  .189  1.000 

Yl:  Fidelity  of 

Task  Judgments 

-.011  .324*  .081  .224*  .318*  .416 

1.000 

Y2:  Fidelity  of  Ac¬ 
tivity  Judgments 

.088  .325*  .011  .235*  .379*  .237 

.615*1.000 

Y3:  Fidelity  of  -.026  .216  -.201  .307*  .367*  .291 

Attribute  Judgments 

.488  *.589  *1.000 

*Significant  at  fi  <  .05. 


Table  3  contains  the  results  of  the  path  analyses.  Both  the 
"constrained"  (paths  to  fidelity  coefficients  were  constrained  to  be 
equal  for  the  three  job  component  models)  and  the  "unconstrained" 
(paths  to  fidelity  coefficients  were  allow  to  assume  different  values) 
models  show  good  fit,  with  the  ratio  of  X2/df  equaling  1.21  and  1.28, 
respectively.  The  Tucker-Lewis  goodness-of-fit  indices  were  .94  and 
.97,  respectively,  indicating  an  adequate  fit  of  the  data  to  the 
model.  The  difference  in  X^s  between  the  constrained  and  unconstrained 
models  was  not  significant.  Multiple  correlations  between  the  judge 
characteristics  and  each  of  the  three  fidelity  coefficients  can  be 
found  in  Table  4.  These  correlations  are  based  on  the  constrained 
model.  R  was  .47  for  the  task-based  fidelity  coefficient  and  .48  for 
the  activity-based  and  attribute-based  fidelity  coefficients, 
indicating  that  the  model  predicts  all  three  fidelity  coefficients 


equally  well.  Figure  2  shows  the  standardized  weights  for  the  paths 
in  the  constrained  model.  The  significant  paths  are: 


t  job  tenure-knowledge  of  job  rated-fidel ity 

•  general  aptitude-knowledge  of  job  rated-fidel ity 

•  general  aptitude-knowledge  own  job-fidelity 

•  general  aptitude-education-knowledge  of  own  job-fidelity. 


Table  3 

Test  of  the  Model's  Goodness-of-Fit 


Model 

X2  df  a 

x2/df 

rho^ 

MSRb 

Model  1:  Yi  -  Y2  =  Y3 
(Constrained) 

1.79  18  .24 

1.21 

.94 

18.80 

Model  2;  Yj  Y?  Y3 
(Unconstrained) 

12.75  10  .24 

1.28 

.97 

16.13 

Difference 

9.04  8  nsC 

^rho  =  Tucker-Lewis  goodness-of-fit  index. 

“Root  Mean  Squared  Residual 

‘^The  two  models  did  not  differ  significantly  (X^  =  9.04,  df=  8,  fi  > 
.05). 

Table  4 

Model's  Prediction  of  Job  Description  Judgment  Fidelity 

Dependent  Variable  Multiple-R 

Fidelity  Coefficient  for  Task-based  Questionnaire  .47 

Fidelity  Coefficient  for  Activity-based  Questionnaire  .48 
Fidelity  Coefficient  for  Attribute-based  Questionnaire  .48 
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Standardized  weights  for  paths  in  the  constrained  model. 


Discussion 


The  purpose  of  the  current  research  was  to  identify 
characteristics  of  individual  raters  who  provide  high-fidelity  job 
description  ratings.  The  following  discussion  addresses  both 
theoretical  and  practical  implications  of  the  research  findings.  An 
important  issue  in  this  investigation  was  whether  the  process  variables 
are  the  same  (constrained  model)  or  different  (unconstrained  model) 
across  different  job  description  instruments.  The  nonsignificant 
difference  found  between  the  two  models  suggest  that  the  constrained 
model  should  be  retained  because  the  model  provided  a  parsimonious 
explanation. 

The  results  of  this  research  support  the  following  conclusions: 

•  the  data  fit  the  model  very  well 

•  the  model  showed  a  high  correlation  between  the  individual 
difference  variables  and  the  fidelity  coefficients 

•  the  same  paths  were  significant  for  all  three  job  description 
instruments. 

The  X^/df  and  rho  criteria  suggest  that  the  data  fit  the  model  very 
well.  The  R's  of  .47  and  .48  are  high  compared  to  the  correlations 
reported  in  the  performance  appraisal  literature. 

Several  tentative  statements  can  be  made  about  the  process 
variables  involved  in  job  description  judgments.  Job  tenure  affects 
rating  fidelity  only  through  its  effect  on  the  knowledge  of  the  rated 
job.  General  aptitude,  on  the  other  hand,  affects  the  fidelity  of 
job  description  in  several  ways: 

•  through  knowledge  of  job  rated 

•  through  knowledge  of  own  job 

•  through  education  and  then  through  knowledge  of  own  job. 

It  is  interesting  to  find  that  general  aptitude  did  nfit  significantly 
relate  to  fidelity  except  through  job  knowledge.  The  direct  path 
coefficients  from  general  aptitude  to  fidelity  were  relative  large 
(.16),  but  not  statistically  significant  for  this  sample.  The  clear 
conclusion  is  that  job  knowledge  is  a  more  direct  determinant  of 
fidelity  than  either  general  aptitude  or  job  experience.  The  degree 
of  independent  contribution  of  general  ability  is  still  somewhat  in 
question,  with  larger  sample  sizes  required  to  confirm  statistical 
significance. 

In  practical  terms,  the  most  important  conclusion  from  the 
current  research  is  that  the  quality  of  job  description  can  be  improved 
by  screening  judges  on  the  basis  of  job  knowledge.  One  benefit  of 
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this  screening  would  be  that  fewer  judges  being  needed  to  meet 
acceptable  levels  of  reliability.  These  findings  also  have 
implications  for  the  collection  of  validity  estimates  provided  by 
expert  judges.  It  seems  plausible  that  both  job  knowledge  and  general 
aptitude  are  required  for  making  reliable  and  valid  estimations  of 
the  validity  of  predictors  for  jobs  or  job  elements. 

Several  cautionary  notes  must  be  struck  concerning  the  present 
research.  First,  the  sample  size  used,  and  the  number  of  jobs  studied 
were  fairly  small.  Second,  the  criteria  focused  on  agreement  between 
judge  and  group  profile  and  did  not  address  the  accuracy/validity  of 
the  group  profile.  Finally,  the  job  experience  and  job  knowledge 
measures  used  were  somewhat  limited.  It  is  quite  possible  that  more 
comprehensive  measures  of  the  judge's  own  job  performance  may  add  to 
the  prediction  of  judgment  accuracy.  Further  research  is  planned  to 
address  these  issues. 
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CHAPTER  8: 


Impact  of  Measurement  Method  on  Standard  Setting  Results 
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SUMMARY 

Most  recent  standard  setting  research  has  involved  determination 
of  minimum  passing  scores  for  multiple  choice  tests.  In  setting  job 
performance  standards,  minimum  levels  must  be  set  for  other  types  of 
measures  such  as  job  performance  ratings  or  work  samples.  This  chapter 
explores  the  relationship  between  the  type  of  performance  measure  used 
and  the  standards  that  are  based  upon  the  measure.  Results  suggest 
that  there  are  clear  differences  in  the  job  performance  standards  set 
by  using  different  methods. 


CHAPTER  8: 


Impact  of  Measurement  Method  on  Standard  Setting  Results 


The  immediate  objective  of  the  SYNVAL  project  is  to  develop  a 
method  of  establishing  prediction  equation  for  MOS  for  which  no 
empirical  measures  of  performance  (yet)  exist.  Under  the  Army's 
current  enlistment  system,  one  more  step  is  required  after  a 
performance  prediction  composite  has  been  defined.  A  minimum 
qualifying  score  must  be  established. 

Currently,  qualifying  scores  are  adjusted  primarily  as  a  function 
of  training  failure  rates,  with  training  success  standards  in  turn 
linked  to  some  aspects  of  on-the-job  performance  as  measured  by  the 
Skill  Qualification  Test  (SQT).  While  some  potential  linkages  between 
enlistment  standards  and  job  performance  exist,  procedures  for  using 
these  linkages  in  setting  standards  are  far  from  uniform  across  MOS, 

The  purpose  of  the  research  report  here  is  to  examine  procedures  for 
setting  job  performance  standards  that  can  be  linked  directly  to 
enlistment  test  scores. 

There  has  been  considerable  research  on  developing  and  evaluating 
procedures  for  setting  standards  in  areas  ranging  from  education  to 
professional  licensure.  Review  of  the  literature  on  setting  standards 
for  Army  jobs  has  been  completed  by  Pulakos  et  al.  (1988).  An 
important  observation  from  the  review  was  that  standard  setting  has 
been  concerned  with  establishing  minimum  scores  for  passing  tests, 
especially  multiple-choice  tests.  Virtually  no  research  has  attempted 
to  apply  standard  setting  methods  to  other  types  of  measures  such  as 
job  performance  measures. 

There  are  a  variety  of  ways  in  which  job  performance  is  measured 
in  different  occupational  fields.  These  include  various  types  of 
scales  for  eliciting  supervisor  ratings  of  performance,  direct  measures 
of  output  (e.g.,  sales  or  units  assembled),  and  work  sample  tests 
(including  assessment  centers).  The  use  of  multiple-choice,  job 
knowledge  tests  is  relatively  rare. 

In  Phase  I  of  the  SYNVAL  project  we  began  to  investigate  ways  of 
obtaining  performance  standards  from  job  experts.  For  the  past  six 
years,  we  have  been  engaged  in  Project  A,  a  massive  project  to  develop 
job  performance  measures  for  different  occupational  specialties  within 
the  Army  (Campbell,  1987).  For  the  purpose  of  obtaining  performance 
standards,  we  designed  three  methods  for  identifying  these  levels  for 
specific  categories  of  performance  as  defined  by  Project  A  measures. 
Army  SMEs  reviewed  performance  level  definitions  and  used  the  three 
methods  for  describing  performance  levels.  We  then  examined  the  level 
of  agreement  among  judges  for  each  method  and  the  degree  of  convergence 
across  the  different  methods. 
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Method 


Project  A  Performance  Measures 

During  the  summer  and  fall  of  1985,  a  total  of  9,470  soldiers  in 
19  different  military  occupational  specialties  (MOS)  underwent  1  to  2 
days  of  job  performance  testing  as  part  of  the  Project  A  Concurrent 
Validation  (CV).  Among  the  different  types  of  job  performance  measures 
were  written  job  knowledge  tests,  hands-on  performance  tests,  and 
supervisor  and  peer  performance  ratings.  The  hands-on  performance 
tests  consisted  of  work  samples  (e.g.,  type  a  letter,  adjust  the  clutch) 
where  possible  and  "walk-through"  simulations  (e.g.,  describe  how  a 
task  is  performed)  where  actual  task  performance  might  be  dangerous, 
expensive,  or  time-consuming.  For  most  tasks,  scores  were  derived  from 
a  checklist  of  critical  steps  each  scored  as  pass  or  fail.  The  ratings 
included  11  common  and  6-12  job-specific  behavioral  summary  scales. 
These  scales  were  developed  through  critical  incident  workshops  that 
generated  examples  of  specific  effective  and  ineffective  behaviors  that 
were  then  sorted  into  dimensions  and  summarized  to  provide  anchors. 

Standard  Setting  Participants 

Our  initial  exploration  of  standard  setting  methodology  includes 
three  of  the  occupational  specialties  from  the  CV.  These  are  71L  ( 
Administrative  Specialist),  63B  (Light-wheeled  Vehicle  Mechanic),  and 
IIB  (Infantryman).  Table  1  shows  the  number  of  workshops  and  number 
of  workshop  participants  for  each  occupational  specialty.  As  shown  in 
this  table,  both  officers  and  enlisted  personnel  were  used  as  judges. 

In  addition,  workshops  were  held  in  both  field  (FORSCOM)  and  training 
(DOTD)  units. 

Standard  Setting  Procedures 

The  focus  of  this  chapter  is  a  comparison  of  standards  based 
on  the  hands-on  test  measures  with  standards  based  on  the  rating 
data.  In  comparing  the  results  from  each  measurement  method,  we 
used  the  CV  sample  data  to  estimate  the  percent  of  current  job 
incumbents  who  pass  each  standard.  For  comparison  purposes,  we 
also  asked  judges  for  a  direct  estimation  of  the  percent  of  current 
incumbents  who  perform  above  or  below  each  standard. 
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Table  1 

Standard  Setting  Participants 


Officers 

Enlisted 

Persons 

Occupational 

Soecialtv 

Type 
of  Unit 

No.  of 
WorkshoDS 

No.  of 
Judoes 

No.  of 
WorkshoDS 

No.  of 
Judoes 

Infantryman 

Field 

2 

35 

2 

42 

Training 

1 

2 

1 

6 

Mechanic 

Field 

2 

11 

2 

16 

Training 

1 

10 

1 

12 

Admin 

Field 

2 

16 

2 

17 

Special ist 

Training 

1 

8 

1 

10 

Standards  to  be  set.  In  comparing  standards  based  on  the 
different  measurement  methods,  we  examined  multiple  standards  for 
several  different  performance  dimensions.  The  different  standards 
were  linked  to  different  possible  organizational  responses  to  the 
performance.  The  performance  levels  differentiated  by  the  different 
standards  were: 


unacceptable  » 


marginal 


acceptable  - 


Soldiers  who  consistently  perform  like 
this  do  not  belong  in  the  Army.  Their 
performance  is  hurting  the  Army,  and 
they  should  be  discharged  early. 

Soldiers  who  consistently  perform  like 
this  need  remedial  training.  Their 
performance  is  of  little  or  no  benefit 
to  the  Army.  Unless  they  receive 
training  and  improve  their  performance, 
they  should  be  barred  from  re¬ 
enlistment. 

Soldiers  who  consistently  perform  like 
this  are  doing  an  adequate  Job.  They 
are  making  positive  contributions  to 
the  Army.  They  should  be  allowed  to 
re-enlist. 
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this  are  doing  extremely  well.  They 
are  making  exceptional  contributions  to 
the  Army  and  are  excellent  examples  to 
their  peers.  They  should  be  encouraged 
to  re-enlist  and  should  be  given 
special  consideration  for  promotion  or 
extra  responsibilities. 


Performance  dimensions.  Judges  were  asked  to  set  standards  for 
several  different  dimensions  of  performance  on  each  job.  Four 
different  performance  dimensions  had  been  identified  as  common  to  all 
Army  jobs,  and  up  to  three  dimensions,  specific  to  each  job 
(Campbell,  McHenry  &  Wise,  1987;  Wise,  Campbell,  McHenry  &  Hanser, 
1986). 

No  job-specific  dimensions  were  selected  for  Infantryman  (IIB), 
because  all  of  their  tasks  can  be  defined  as  general  soldiering 
tasks.  For  mechanics  and  administrative  specialists,  we  identified 
job  components  from  the  Task-Based  Model  that  were  rated  as  important 
by  SMEs  in  the  job.  We  grouped  these  components  so  that  they  could 
be  more  easily  matched  to  specific  hands-on  task  tests  and  to 
individual  job  component  rating  scales.  Figure  1  shows  the 
components  with  a  mean  importance  rating  of  4.0  (very  important)  or 
better.  Figure  2  lists  both  the  general  and  job-specific  performance 
dimensions  that  we  selected,  and  shows  which  dimensions  were  used  for 
each  of  the  three  standard  setting  methods  described  below. 
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V 


Job-Specific  Task  Components 

Standard  Setting 

with  "High"  or  "Extremely  High" 

Performance 

Mean  Importance  Ratings 

Dimensions 

Mean 

MOS 

Rating  Component 

63B 

4.2 

Vehicle  Main/Check/Services 

Basic  Maintenance 

4.1 

Troubleshoot  Mech.  System 

4.2 

Operate  Wheeled  Vehicle 

Vehicle  Recovery 
and  Operation 

71L 

4.8 

Type 

Typing 

4.3 

Record,  File  Dispatch 

Information 

Filing/Other 

Clerical 

4.0. 

Prepare  Tech.  Forms  and 

Documents 

Figure  1.  Selection  of  performance  dimensions  for  Phase  I  standard 
setting. 
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Performance 

Soldier- 

Based 

Incident- 

Based 

Task-Based 

Dimension 

IIB 

63B 

71L 

IIB 

638 

71L 

IIB  63B  71L 

General 

Soldiering 

X 

X 

X 

X 

X 

XXX 

Effort  and 
Leadership 

X 

X 

X 

X 

X 

X 

Personal 

Discipline 

X 

X 

X 

X 

X 

X 

Physical 

Fitness  and 

Military 

Bearing 

X 

X 

X 

X 

X 

X 

Basic  Main¬ 
tenance 

X 

X 

X 

Recover/ 

Vehicle 

Operation 

X 

X 

Administration 
and  Plan¬ 
ning 

X 

X 

Typing 

X 

X 

X 

Fil ing/Other 
Clerical 

X 

X 

X 

Figure  2.  Performance  dimensions  used  with  each  standard  setting 
method. 
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Variables 


Task-based  method.  Ten  different  levels  of  performance  were 
defined  by  dividing  the  sample  of  soldiers  tested  into  deciles  based 
on  their  overall  hands-on  score  for  each  dimension.  Two  to  five 
tasks  were  selected  for  each  performance  dimension  and  average  scores 
(generally  percent-GO)  were  computed  for  each  decile  group.  For  each 
of  the  ten  performance  levels,  judges  were  told  the  average  score  for 
each  task  and  also  the  percent  passing  critical  steps  in  each  of  the 
sample  tasks.  Judges  were  then  asked  to  rate  the  acceptability  of 
each  of  the  ten  performance  level  descriptions  using  the  four-point 
scale  described  above  (Unacceptable  to  Outstanding).  Judges  also 
were  asked  to  indicate  the  minimum  percent  correct  for  each  of  the 
performance  levels.  Figure  3  shows  an  example  of  the  rating  forms 
used  in  this  exercise.  Figure  4  shows  an  example  of  supplemental 
information  about  critical  steps  that  was  provided  to  the  judges. 

Critical  incident-based  method.  In  developing  anchors  for  the 
different  rating  scales,  several  hundred  critical  incidents  (behavioral 
examples)  were  collected  and  scaled  with  respect  to  the  level  of 
effectiveness  that  they  indicated.  We  selected  between  24  and  69  of 
these  incidents  for  each  performance  dimension,  spaced  throughout  the 
range  of  effectiveness.  Judges  were  asked  to  rate  the  acceptability 
of  the  performance  indicated  in  each  of  the  se’wv.ucd  incidents  using 
the  same  four-level  scale  of  performance  acceptability.  Figure  5 
shows  an  example  of  this  exercise.  Cutting  points  were  defined  as 
the  midpoint  between  the  most  effective  incident  judged  to  be  in  one 
acceptability  category  and  the  least  effective  incident  judged  to  be 
in  the  next  highest  category. 

Soldier-based  method.  For  each  performance  dimension,  judges  were 
asked  to  estimate  the  percent  of  current  job  incumbents  at  each  level 
of  performance.  Figure  6  shows  and  example  of  the  exercise  used  to 
obtain  soldier-based  judgments. 
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PercenH  oi  Staps  Passed  for  Soldiers  at 
Diffarant  Perfomance  Levels 


TVf 

r 

LZVS  Lr/6  LZ!^7  1-73 

IV'? 

L-/10 

Task 

61 

70 

73  76 

73  80 

82  84 

87 

90  • 

Average  for  Scc.h  Tasks 

57 

63 

70  72 

74  76 

73  80 

83 

36 

Apply  Pressure  Dressing 

65 

72 

77  81 

83  35 

87  89 

91 

93 

Lead,  Reduce,  Clear  hoO 

. 

Yeur  Pati 

.no  of  this 

ormance 

Level : 

i  steps 

Passed  gnacrartabla  Marginal  Acser  table  Outstar.dinc 


1.  61%  _  _  _  _ 

2.  70%  _  _  _  _ 

3.  73%  _  _  _  _ 

4.  76%  _  _  _  _ 

5.  73%  _  _  _  _ 

6.  80%  _  _  _  _ 

7.  32%  _  _  _  _ 

3,  84%  _  _  _  _ 

9.  87%  _  _  _  _ 

10.  90%  _  _  _  _ 

1.  What  is  the  miniaua  percent  cf  steps  passed 

that  you  would  rate  as  marci.’-.al?  _ i 

2.  What  is  the  miniaua  percent  of  steps  passed 

that  you  would  rata  as  acceatabla?  _ 

3 .  What  is  the  miniaua  percent  of  steps  passed 

that  you  would  rata  as  outstanding?  _ i 


Figure  3.  Task-based  approach  to  standard  setting. 
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EZAMPLSS  07  DimCULT  HANDS-ON  TEST  STE7S 
IIB:  General  Soldiering  Tasks 

Fora  A 

Here  are  examples  of  the  more  difficult  hands-on  test  steps. 
These  steps  were  often  failed  by  poor  performers,  but  were 
usually  passed  by  the  best  performers. 

For  each  of  the  lo  performance  levels,  we  show  the  percent  of 
soldiers  at  the  level  who  correctly  perform  steps  like  those 
shown.  In  the  first  example,  75%  of  the  soldiers  at  performance 
level  1  maintained  the  sterility  of  the  dressing  properly.  At 
performance  level  9,  96%  of  the  soldiers  performed  this  step 
correctly. 


Percent  of  Soldiers  at  Each  Performance  Level 
Who  Pass  Specific  Steps  in  the  Hands-on  Tests 


Hands-On  Test  Stems 

'll 

ID 

•1 

'formance 

Level 

-1  -2. 

_3 

4 

_5 

6 

7 

8 

9 

10 

Aool 

.v  Pressure  Dres^ipq 

5. 

Maintained  sterility  of  dressing. 

73 

87 

89 

91 

92 

93 

94 

95 

96 

97 

6. 

Tied  the  tails  in  a  non-slip 
knot  (square  knot) . 

39 

53 

53 

63 

69 

71 

72 

74 

76 

S3 

3  . 

Sealed  edges  of  dressing  with  the 
tails  (with  at  least  1/2'*  overlap)  . 

23 

31 

33 

36 

39 

42 

43 

49 

53 

37 

Prepare  a  Draeon  for  Firing 

3. 

Correctly  secured  tracker 
receptacle  cover. 

26 

32 

33 

43 

48 

55 

62 

69 

76 

33 

10. 

Correctly  locked  tracker  in  place 

59 

62 

69 

73 

77 

82 

85 

83 

91 

94 

14. 

Completed  preparing  the  Dragon 
within  2  minutes. 

63 

71 

74 

77 

80 

83 

86 

89 

93 

96 

Load.  Reduce,  Clear  M60 

14. 

Pulled  cocking  handle  to  the  rear, 
locking  the  bolt  to  rear.  (To 
eject  round  after  stoppage.) 

74 

81 

91 

93 

94 

95 

96 

97 

93 

99 

23  . 

Placed  safety  in  SAFE  after 
clearing. 

58 

63 

75 

80 

82 

86 

87 

83 

90 

95 

29. 

Performed  clear  steps  in  proper 
sequence . 

23 

30 

49 

54 

59 

63 

68 

73 

77 

82 

Figure  4.  Examples  of  difficulty  hands-on  test  steps. 
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CRITICAL  INCIDENT-BASED  APPROACH 
TO  STANDARD  SETTING 


Z2:^'i?LZ 


This  soldier  spent  many  duty  and  non-dun' 
hours  lenmini  his  new'MOS.  In  a  few 
months,  he  was  tops  in  his  MOS  and  was 
seiected  as  the  first  E-4  to  evaluate  other 
soldiers  in  the  MOS. 

The  rater  read  the  exanple  and  noticed  that  only  1%  of 
InfantrTtaen  consistently  periora  as  well  as  or  better  than 
the  soldier  in  the  exa'aple.  The  rater  decided  the 
perforaance  of  the  soldier  in  this  exaaple  was  outstanding, 
and  placed  an  ”X'*  under  Outstanding. 


Figure  5.  Critical  incident-based  approach  to  standard  setting. 
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SOLDIER-BASED  APPROACH 
TO  STANDARD  SETTING 


EXAMPLE 


EPPORT  AlID  L2RBSRSHIP 

This  rafars  to  hew  hard  a 
soldiar  tries  on  the  job, 
especially  under  adverse  or 
dangerous  conditions.  It  also 
refers  to  the  support  and 
leadership  a  soldier  provides 
for  peers. 


Unacceptable 

Marginal 

Acceptable 

Outstanding 

TOTAL 


iO _ % 

JO  % 
TT _ % 

_ % 

100% 


The  rater  in  this  case  was  a  94B  NCO.  Ke  read  the 
definition  of  Effort  and  Leadership,  then  thought  about  the 
cooks  with  18  months  on-the-job  exper±hnce  that  he  was 
supervising.  He  decided  10%  were  doing  an  unacceptable  job, 
10%  were  doing  a  marginal  job,  75%  were  doing  an  acceptable 
job,  and  5%  were  doing  an  outstanding  job  on  Effort  and 
Leadership.  He  then  checked  his  percents  to  make  sure  they 
added  up  to  100%. 


Figure  6.  Soldier-based  approach  to  standard  setting. 
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Results 


Army  SMEs  found  the  four  performance  level  definitions  to  be 
reasonable.  Most  reported  that  the  definitions  were  clear  and 
practical.  However,  a  few  SMEs  expressed  some  reservations  for  the 
consequence  of  "unacceptable."  They  questionned  if  barring  a  soldier 
from  re-enlisting  might  be  too  harsh  for  a  performing  a  task 
unacceptably. 

Table  2  shows  the  percent  of  current  job  incumbents  with  performance 
classified  as  outstanding  for  each  performance  dimension  based  on 
standards  from  each  of  the  three  methods.  Table  2  also  shows  the 
standard  deviation  across  judges  in  these  percents.  For  the  soldier- 
based  method,  these  percentages  were  supplied  by  the  judges.  For  the 
other  two  methods,  these  percentages  were  estimated  by  applying  the 
cutting  points  supplied  by  each  judge  to  distributional  information 
from  the  CV  sample. 

Table  2 

Methods  of  Judging  Implied  Percent  of  Soldiers  Performing  at  Each 
Level 


Percent  Percent 

Occupational  Performance  Unacceptable  Outstanding 


Soecialtv 

Dimension 

Method 

N 

Mean 

SD 

Infantryman 

General 

Soldier 

80 

8.0 

5.3 

12.4 

9.6 

Soldiering 

Task 

81 

21.0 

14.9 

7.7 

9.4 

Ratings 

80 

6.3 

13.3 

11.6 

15.0 

Mechanic 

General 

Soldier 

49 

8.4 

6.9 

16.3 

18.6 

Soldiering 

Task 

50 

23.0 

14.6 

11.0 

12.1 

Basic 

Soldier 

49 

12.6 

12.8 

11.0 

10.5 

Maintenance 

Task 

50 

6.0 

7.4 

34.4 

20.8 

Ratings 

49 

4.4 

16.3 

8.8 

12.6 

Clerk 

General 

Soldier 

47 

10.7 

10.5 

10.7 

9.7 

Soldiering 

Task 

51 

18.9 

12.6 

11.9 

11.6 

Typing 

Soldier 

47 

8.1 

5.5 

12.0 

13.8 

Task 

51 

35.7 

15.6 

7.3 

7.6 

Incident52 

10.8 

14.7 

9.2 

12.2 

Other 

Soldier 

47 

10.3 

13.0 

10.8 

14.4 

Clerical 

Task 

50 

35.7 

18.7 

8.0 

7.9 

Incident52 

4.6 

12.4 

4.8 

5.6 
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The  different  methods  of  identifying  job  performance  standards 
led  to  significantly  different  results.  The  critical  incident-based 
method  resulted  in  very  lenient  standards  for  minimal  performance 
with  very  few  soldiers  falling  in  the  unacceptable  category.  By 
comparison,  the  task-based  method  led  to  considerably  higher  failure 
rates.  The  direct,  soldier-based,  approach  led  to  estimates  of 
failure  rates  that  were  generally  between  these  two  extremes. 

At  the  upper  end  of  the  scale,  the  critical  incident-based 
method  led  to  very  strict  standards,  with  relatively  few  soldiers 
classified  as  outstanding.  The  overall  effect  of  the  critical 
incident-based  approach  was  a  compressed  scale  with  most  job  incumbents 
classified  as  either  marginal  or  acceptable.  The  task-based  method 
had  mixed  results  at  the  upper  end,  but  on  average  also  showed 
relatively  few  incumbents  classified  as  outstanding  in  comparison  to 
the  soldier-based  results.  The  overall  effect  of  the  task-based 
approach  was  relatively  strict  standards. 

The  task-based  approach  led  to  greater  variation  across  judges 
in  comparison  to  the  other  two  methods.  The  critical  incident-based 
approach  had  slightly  greater  agreement  (lower  variance)  in  comparison 
to  the  direct  estimates  of  soldier  performance. 

Discussion 


There  was  general  agreement  among  SMEs  that  the  four  performance 
level  definitions  (for  unacceptable,  marginal,  acceptable,  and 
outstanding)  were  useful  distinctions.  Results  also  supported  that, 
for  each  standard  setting  method,  the  SMEs  could  make  meaningful 
descriptions  according  to  the  four  definitions.  However,  the  three 
different  measurement  methods  did  lead  to  different  standards  and 
also  to  some  difference  in  the  degree  of  consensus  that  was  achieved 
in  setting  the  standards.  The  critical  incident-based  method  yielded 
results  that  were  generally  consistent  with  the  judges  direct 
estimation  of  the  distribution  of  job  performance  in  comparison  to 
the  task-based  method.  These  results  provide  support  for  further 
exploration  of  the  critical  incident-based  method  which  used  specific 
behavioral  examples  as  anchors  in  describing  different  levels  of  job 
performance. 
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CHAPTER  9: 


Combining  Individual  Standards  into  an  Overall  Standard: 
Modeling  the  Judgment  Process  and  Investigating  Differences 

among  Judges 


Philip  L.  Szenas 
Lauress  L.  Wise 

American  Institutes  for  Research 


SUMMARY 

This  chapter  examined  the  conjoint  measurement  approach  to 
setting  overall  job  performance  standards.  The  conjoint  measurement 
approach  attempts  to  mathematically  model  the  qualitative  laws  that 
judges  use  to  combine  information  and  make  judgments. 

Judges  were  provided  information  about  the  component  level  job 
performance  of  40  hypothetical  soldiers.  These  judges  were  then 
asked  to  combine  this  information  into  an  overall  job  performance 
judgment.  The  overall  job  performance  standards  were  then  regressed 
onto  rescaled  values  of  the  component  standards  in  order  to  model  the 
judges'  "policies."  Group  differences  in  judgment  strategies  were 
also  investigated. 

The  results  of  these  analyses  suggest  that:  (1)  the  average 
judgments  were  highly  linear,  but  a  more  accurate  model  shows  that  a 
configural  effect  was  found  (leniency;  standard  deviation  of 
components);  and  (2)  that  different  judge  strategies  were  found 
across  rank,  across  command,  and  across  job. 


CHAPTER  9; 


Combining  Individual  Standards  into  an  Overall  Standard: 
Modeling  the  Judgment  Process  and  Investigating  Differences 

among  Judges 


Much  of  the  literature  on  standard  setting  concerns  a  single 
measure  or  a  single  dimension  of  performance.  Project  A  and  the  Army 
Synthetic  Validation  Project,  however,  take  a  multidimensional 
perspective  of  job  performance.  A  central  issue  to  be  considered 
when  taking  a  multidimensional  approach  is  that  an  employee's  job 
performance  may  be  quite  satisfactory  in  some  areas,  but  not 
satisfactory  in  others.  Thus,  decisions  must  be  made  regarding  the 
extent  to  which  more  effective  performance  in  some  areas  compensates 
for  less  effective  performance  in  others.  These  decisions  will 
dictate  how  standards  for  individual  dimensions  of  performance  should 
be  combined  into  an  overall  performance  standard. 

The  question  of  how  to  set  an  overall  standard  for  job  performance 
must  necessarily  be  preceded  by  the  development  of  a  scale  for 
assessing  overall  job  performance.  Several  different  approaches  for 
developing  such  an  overall  performance  scale,  ranging  from  a  simple 
linear  composite  to  more  complex  conjoint  measurement  techniques, 
were  examined  as  part  of  the  Project  A  research  (Sadacca,  Park  & 

White,  1986).  A  conjoint  measurement  approach  (e.g.,  Luce  &  Tukey, 
1964;  Green  i  Srinivasan,  1978)  asks  judges  to  evaluate  tradeoffs 
among  increments  and  decrements  along  different  dimensions.  For 
example,  two  soldiers,  one  having  a  slightly  higher  level  of 
proficiency  and  a  slightly  lower  level  of  motivation  than  the  other, 
might  be  compared  in  terms  of  their  overall  contribution  to  the 
organization. 

In  its  general  form,  the  conjoint  measurement  model  would  not 
assume  that  the  value  of  a  performance  increment  is  necessarily  the 
same  for  different  parts  of  different  dimensions.  It  is  possible, 
for  example,  that  small  decrements  below  minimum  levels  in  some  areas 
are  balanced  only  by  large  increments  above  minimum  levels  in  other 
areas.  There  are  two  special  cases  of  interest  in  setting  an  overall 
performance  standard.  In  the  first  case,  no  amount  of  increment  in 
other  areas  can  compensate  for  below  standard  performance  on  any 
other  dimension.  Using  this  model,  known  as  the  multiple  hurdles 
model .  an  examinee  fails  the  overall  standard  if  he  or  she  fails  any 
of  the  individual  standards.  The  other  special  case  of  interest  is  a 
strictly  linear  model,  when  overall  performance  is  measured  by  a 
weighted  sum  of  the  individual  performance  measures.  Using  this 
model,  known  as  the  compensatory  model .  a  decrement  in  one  performance 
area  could  be  compensated  for  by  an  equal  increment  in  another  area. 

This  paper  examines  the  conjoint  measurement  approach  to  setting 
overall  job  performance  standards.  The  conjoint  measurement  approach 


attempts  to  mathematically  model  the  qualitative  laws  that  judges  use 
to  combine  information  and  make  judgments.  The  advantage  of  this 
approach  is  that  it  allows  for  nonlinear  variations  of  the  multiple 
hurdles  and  compensatory  models  to  be  discovered.  The  disadvantage 
of  this  approach  is  that  it  relies  on  the  ability  of  the  judges  to 
combine  multiple  sources  of  information  in  order  to  make  judgments. 
The  general  procedure  is  as  follows:  judges  are  provided  with 
information  about  performance  standards  on  several  job  dimensions  and 
are  asked  to  combine  this  information  and  provide  an  overall  job 
performance  standard.  A  mathematical  model  is  then  constructed  to 
capture  the  judge's  policy.  This  model  is  then  used  to  transform 
information  on  individual  dimensions  into  an  overall  score  that  can 
be  compared  with  the  overall  standard. 

If  we  employ  the  conjoint  measurement  approach,  we  must  be  aware 
of  some  of  the  findings  on  decision  making  and  social  cognition. 
Research  on  decision  making  has  demonstrated  the  power  of  the  linear 
model  in  explaining  conjoint  judgments.  Slovic  and  Lichtenstein 
(1971)  have  conducted  a  comprehensive  review  of  the  approaches  to  the 
study  of  information  processing  in  judgments  and  showed  that  although 
judges  have  difficulty  weighting  and  combining  information,  their 
responses  are  highly  predictable  using  the  linear  model.  Yntema  and 
Torgerson  (1961;  cited  in  Slovic  i  Lichtenstein,  1971)  demonstrated 
that  94%  of  the  variap'-.e  of  a  truly  configural  function  could  be 
predicted  from  an  aCjitive  combination  of  main  effects.  In  research 
using  the  conjoin*  measurement  technique  for  Project  A,  Sadacca  et 
al.  (1986)  found  that  linear  composites  provided  reasonable 
approximations  to  the  conjoint  scaling  results. 

There  is  also  research  in  social  cognition  that  suggests  that 
extreme  information  has  a  tendency  to  become  particularly  salient  and 
receive  a  greater  weight  when  used  in  combination  with  other 
dimensions  in  order  to  form  an  overall  evaluation.  For  example, 

Fiske  (1980)  found  that  a  extreme  stimuli  are  more  salient  than 
moderate  stimuli.  It  is  quite  possible  that  judges  may  provide  lower 
ratings  for  ratees  who  show  either  a  single  extreme  value  on  a 
dimension,  or  a  wide  variation  in  behavior  across  dimensions. 

Keeping  these  issues  in  mind,  the  purpose  of  this  paper  is  to 
develop  a  mathematical  model  that  captures  the  strategy  that  judges 
use  when  combining  individual  standards  on  three  job  performance 
dimensions  into  an  overall  standard.  These  three  dimensions  are: 

Core  Technical  Proficiency  (CTP;  performance  on  core  job-specific 
tasks).  Effort  and  Leadership  (ELS),  and  Maintaining  Personal 
Discipline  (MPD).  These  dimensions  were  developed  rationally  and 
confirmed  empirically  for  19  diverse  jobs  as  a  part  of  large 
criterion  development  effort  (Wise,  Campbell,  McHenry,  &  Hanser, 
1986).  We  will  compare  the  strict  compensatory  model  vs.  the  strict 
multiple  hurdles  model,  and  several  alternative  intermediate  models. 
We  will  address  the  issues  concerning  the  use  of  judges  to  combine 
multiple  sources  of  information  for  making  judgments.  We  will  also 
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investigate  the  notion  that  different  judge  characteristics  may  be 
correlated  with  different  strategies. 


Method 


Subjects 

One  hundred  eighty-one  judges  from  five  sites  and  representing 
three  military  occupational  specialties  (MOS)  provided  the  standard 
setting  judgments.  The  judges  consisted  of  93  Non-Commissioned 
Officers  (NCOs),  74  Officers,  and  11  Civilians;  there  were  156  males 
and  22  females.  Table  1  shows  demographic  information  for  the 
sample.  Regression  equations  were  computed  for  each  judge  by 
regressing  overall  judgments  onto  the  three  performance  dimensions. 
Mean  ratings  for  the  three  regression  weights  (CTP,  ELS,  &  MPD), 
along  with  the  mean  intercept  value,  were  used  to  screen  outliers. 
Two  judges  were  eliminated  from  further  analyses.  The  first  judge's 
regression  weight  for  the  CTP  dimension  was  4.2  standard  deviations 
above  the  mean.  The  second  judge's  regression  weight  for  the  MPD 
dimension  was  5.2  standard  deviations  above  the  mean.  These  judges 
were  two  white  males,  one  was  an  Infantry  Officer  from  a  field 
(FORSCOM)  site  while  the  other  was  a  Vehicle  Mechanic  NCO  from  a 
training  (DOTD)  site. 

Procedure 


The  data  were  collected  in  a  series  of  five  site  visits  during  the 
months  of  March  and  April,  1988.  Each  workshop  lasted  six  hours  and  the 
exercise  described  in  this  paper  was  always  the  last  exercise  of  the  one- 
day  workshops. 

A  conjoint  measurement  approach  was  used  to  determine  how  the  judges 
evaluated  the  tradeoffs  between  different  increments  and  decrements  of 
performance  on  different  dimensions  when  setting  overall  performance 
standards.  Subjects  were  provided  information  on  40  hypothetical  soldiers 
that  varied  in  their  performance  on  three  dimensions:  Core  Technical 
Proficiency  (CTP;  performance  on  MOS-specific  tasks).  Effort  and 
Leadership  (ELS),  and  Maintaining  Personal  Discipline  (MPD).  The 
soldier's  performance  on  each  dimension  was  described  as  unacceptable  (U), 
marginal  (M),  acceptable  (A),  or  outstanding  (0).  The  judges  were  asked 
to  provide  an  overall  performance  rating  (Rating  Scale:  U  -  unacceptable, 
M  -  marginal,  A  -  acceptable,  0  -  outstanding)  for  each  of  the  40 
hypothetical  soldiers.  For  example,  hypothetical  soldier  number  13  was 
marginal  on  the  core  technical  dimension,  unacceptable  on  the  Effort  and 
Leadership  dimension,  and  outstanding  on  the  maintaining  personal 
discipline  dimension.  A  copy  of  the  research  instrument  can  be  found  in 
Appendix  F. 
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Table  1 

Demographic  Information  On  the  Judge  Sample 


Demographic 

Variable  N 


MOS 


IIB:  Infantryman 

81 

63B:  Light  Vehicle  Mechanic 

46 

71L:  Clerk  Typist 

52 

Rank 

Non-Commissioned  Officer  (NCO) 

93 

Officer 

74 

Civilian 

11 

Command 

Training  Site  (DOTD) 

55 

Field  Site  (FORSCOM) 

124 

Gender 

Female 

22 

Male 

156 

Race 

Black 

45 

American  Indian 

1 

Hispanic 

15 

White 

104 

Other 

13 

Results 


Reliability  of  Judgments 

A  variance  component  analysis  of  the  mean  ratings  of  the  40 
hypothetical  soldiers  by  the  179  judges  showed  a  single-rater 
reliability  of  .66. 

Scaling 

The  first  issue  addressed  in  the  analyses  was  how  to  scale  the 
ratings.  It  was  believed  that  the  psychological  distance  between 
unacceptable  and  marginal,  for  example,  could  be  different  from  the 
distance  between  acceptable  and  outstanding.  In  the  first  step,  the 
responses  were  simply  scaled  as  Integers  (1,  2,  3,  and  4  representing 
the  response  options:  unacceptable,  marginal,  acceptable,  and 
outstanding).  Regression  equations  were  then  computed  using  the  mean 
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rating  for  each  of  the  40  hypothetical  soldiers  across  all  judges  and 
the  integer  scaling  of  each  dimension.  The  results  of  this  analysis 
suggested  that  a  better  scaling  of  the  performance  levels  could  be 
achieved.  When  a  nonlinear  regression  model  was  used  to  fit  the 
data,  the  increased  from  .939  to  .963.  The  results  of  the  nonlinear 
analysis  produced  four  different  point  estimates  (.66,  1.95,  2.94, 
and  3.62)  and  showed  that  the  psychological  distance  between  the 
performance  levels  were  different  (i.e.,  unacceptable-marginal  = 

1.29,  marginal -acceptable  »  .99,  and  acceptable-outstanding  =  .68). 

Both  the  original  integer  values  and  the  revised  scale  values  were 
used  in  the  remainder  of  the  analyses. 

Evaluation  of  Different  Models 


Four  models  were  investigated  using  both  the  integer  scale 
values  and  the  rescaled  values  described  above,  for  a  total  of  eight 
approaches  to  the  modeling  of  the  judges'  policy.  The  results  of 
these  analyses  can  be  found  in  Table  2.  A  more  detailed  description 
of  the  four  models  using  the  rescaled  values  follows. 

Pure  multiple  hurdles  model.  In  order  to  determine  if  judges 
used  a  pure  multiple  hurdles  model,  the  mean  ratings  for  the  40 
hypothetical  soldiers  were  regressed  onto  the  lowest  value  across  the 
three  dimensions.  This  model  assumes  that  a  soldier's  overall 
evaluation  is  equal  to  his  or  her  lowest  dimension  score.  The  results 
of  this  analysis  showed  that  =  .589,  fi  <  .01,  for  the  model  with 
the  regression  coefficient  equal  to  .61,  fi  <  .01  and  the  intercept 
equal  to  1.30,  e  <  .01. 

Purely  compensatory  model  with  unit  weights.  This  model  has  two 
assumptions:  First,  that  any  decrement  in  performance  on  one  dimension 
can  be  compensated  for  by  an  equal  increment  in  another  dimension. 
Second,  each  dimension  of  job  performance  contributes  equally  to 
overall  job  performance.  This  model  was  evaluated  by  regressing  the 
mean  ratings  onto  a  simple  additive  composite  of  the  three  dimensions. 
The  r2  for  this  model  was  .963,  £  <  .01.  The  coefficient  for  the 
composite  was  .35,  fi  <  .01,  and  the  intercept  for  the  model  was  -.25, 

E  <  .01. 

Compensatory  model  with  differential  weights.  This  model  also 
assumes  a  compensatory  relationship  among  dimensions  but  it  allows 
the  separate  dimensions  to  differ  in  their  contribution  to  the  overall 
standard.  For  the  evaluation  of  this  model,  the  mean  ratings  were 
regressed  onto  the  three  job  performance  dimensions.  The  results 
showed  an  R^  -  .968,  e  <  -01 •  The  weights  for  the  three  regression 
variables  and  intercept  were  .37,  .31,  .37,  and  -.25  respectively. 

All  estimates  were  significant  at  e  <  .01. 


Table  2 


Comparison  of  the  Different  Models:  Regression  Coefficients  and  R^s 


Single 

Model  CTP  ELS  MPD  Coefficient  Intercept  Std.  Dev. 


1:  Pure  Multiple  .  .69 

Hurdle  w/ 

Integer  Scale 
Values 

2:  Pure  Multiple  .  .61* 

Hurdle  w/ 

Rescaled  Values 

3:  Compensatory  .  .34* 

w/Unit  Weights 
&  Integer  Scale 
Values 

4:  Compensatory  .  .36* 

w/Unit  Weights 
&  Rescaled  Values 

5:  Compensatory  .33*  .32*  .35*  -- 
w/Oifferential 
Weights  &  Integer 
Scale  Values 

6:  Compensatory  .37*  .31*  .37*  -- 
w/Differential 
Weights  & 

Rescaled  Values 

7:  Nonlinear  .33*  .29*  .34*  -- 

Model  w/Penalty 
&  Integer  Scale 
Values 

8:  Nonlinear  .37*  .30*  .36*  -- 

Model  w/Penalty 
&  Rescaled  Values 


1.16*  --  .650* 

1.30*  --  .588* 

-.22*  --  .939* 

-.25*  --  .963* 

-.19*  --  .940* 

-.25*  --  .968* 

.12  -.23*  .969* 

-.09  -.11*  .974* 


*Estimates  are  significant  at  <  .01. 
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Table  3 


Comparison  of  the  Eight  Models:  Repression  Results  and  F-tests 


Model 

Source 

DF 

SS 

MS 

F 

E  < 

1:  Pure  Multiple  Hurdle  w/ 

Model 

1 

11.40 

11.40 

70.41 

.01 

Integer  Scale  Values 

Error 

38 

6.15 

0.16 

Total 

39 

17.56 

2:  Pure  Multiple  Hurdle  w/ 

Model 

1 

11.40 

11.40 

54.35 

.01 

Rescaled  Values 

Error 

38 

7.97 

0.21 

Total 

39 

19.79 

3:  Compensatory  w/  Unit 

Model 

1 

16.48 

16.48 

583.35 

.01 

Weights  &  Integer  Values 

Error 

38 

1.07 

0.03 

Total 

39 

17.55 

4:  Compensatory  w/  Unit 

Model 

1 

18.65 

18.65 

982.83 

.01 

Weights  &  Rescaled  Values 

Error 

38 

0.72 

0.02 

Total 

39 

19.37 

5;  Compensatory  w/  Differential 

Model 

3 

16.51 

5.50 

189.43 

.01 

Weights  &  Integer  Values 

Error 

36 

1.05 

0.03 

Total 

39 

17.56 

6:  Compensatory  w/  Differential 

Model 

3 

18.75 

6.25 

365.04 

.01 

Weights  &  Rescaled  Values 

Error 

36 

0.62 

0.02 

Total 

39 

19.37 

7:  Nonlinear  Model  w/Penalty 

Model 

4 

17.02 

4.25 

276.11 

.01 

&  Integer  Values 

Error 

35 

0.54 

0.02 

Total 

39 

17.56 

8:  Nonlinear  Model  w/Penalty 

Model 

4 

18.87 

4.72 

327.46 

.01 

&  Rescaled  Values 

Error 

35 

0.50 

0.01 

Total 

39 

19.37 

Nonlinear  model:  Penalty  for  inconsistent  or  extreme  behavior. 
This  model  allows  for  variation  in  behavior  to  enter  the  equation. 
According  to  the  linear  model,  extreme  decrements  in  performance  on 
one  dimension  can  be  compensated  for  by  extreme  increments  in 
performance  on  another  dimension.  The  linear  model  also  assumes  that 
decrements  on  several  dimensions  can  be  compensated  for  by  increments 
on  several  dimensions  or  extreme  increments  on  a  few  dimensions.  The 
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nonlinear  model  suggests  that  the  overall  evaluation  is  a  function  of 
not  only  the  average  performance  across  dimensions,  but  also  the 
consistency  of  performance  across  dimensions.  This  model  was  tested 
by  including  the  standard  deviation  of  the  three  performance  dimensions 
into  the  regression  equation.  If  the  coefficient  for  the  standard 
deviation  variable  was  significant,  this  would  suggest  a  nonlinear 
relationship  between  the  three  dimensions  and  the  overall  standard. 

The  for  this  model  was  .974,  fi  <  .01.  The  regression  coefficients 
were  .37,  fi  <:  .01  for  CTP,  .30,  fi  <  .01  for  ELS,  .36,  e  <  .01  for 
MPD,  and  -.09,  e  <  .01  for  the  intercept.  The  estimate  for  the 
standard  deviation  variable  was  -.11,  e  <  ♦01*  Table  2  contains  the 
regression  weights  and  the  R^s  for  the  different  models.  Table  3 
shows  the  F-tests  for  the  significance  of  each  model. 

Tables  4  and  5  test  for  differences  among  the  models.  Table  4 
tests  for  differences  between  the  four  models  that  used  integer  scale 
values  and  the  four  models  using  the  revised  scale  values.  In  all 
cases,  except  the  multiple  hurdles  model,  the  revised  scaled  models 
had  significantly  larger  R^s.  Table  5  tests  for  differences  among 
the  two  compensatory  models  and  the  nonlinear  model  for  the  revised 
scale  values  only.  The  multiple  hurdles  model  could  not  be  compared 
with  the  other  models  because  the  explicit  assumption  of  the  model 
did  not  fit  the  assumptions  of  the  other  three  models.  The  table 
shows  that  the  compensatory  model  with  differential  weights  had  a 
significantly  larger  R^  than  the  compensatory  model  with  unit  weights. 
The  table  also  shows  that  the  nonlinear  model  with  penalty  had  a 
significantly  larger  R^  than  either  of  the  two  compensatory  models. 


Rater  Characteristics  and  Rater  Strategy 

Cluster  analysis.  A  cluster  analysis  was  conducted  to  investigate 
whether  judges  could  be  classified  into  different  judgment  strategies. 
The  variables  used  to  cluster  the  judges  were  the  regression  weights 
for  CTP,  ELS,  and  MPD  and  the  intercept  from  each  judge's  individual 
regression  equation.  The  results  of  the  cluster  analysis  supported  a 
three  cluster  solution.  The  mean  regression  equations  across  judges 
within  cluster  can  be  found  in  Table  6.  Figure  1  shows  a  plot  of 
these  regression  equations.  Cluster  1  was  composed  of  judges  who 
were  strict  in  setting  overall  standards.  The  large  negative  intercept 
value  for  this  group  means  that  their  overall  ratings  were  lower  than 
a  weighted  average  of  the  individual  ratings.  In  contrast.  Cluster  3 
appears  to  be  made  up  of  judges  whose  standards  were  lenient.  This 
was  evident  by  a  positive  intercept  value.  Cluster  2,  by  far  the 
largest  group,  was  composed  of  judges  that  used  a  linear  approach 
with  slightly  different  weights  for  the  three  dimensions. 
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Table  4 


Comparison  Among  Four  Models:  Integer  vs.  Noninteger  Scaling 


Comparison 

Source 

DF 

SS 

MS 

F  £  < 

Models  1  & 

2 

Total 

179 

179.00 

Error 

174 

73.79 

0.42 

Model  1 

2 

105.25 

Model  2 

5 

116.35 

Difference 

3 

11.10 

3.70 

8.81  .01 

Models  3  i 

4 

Total 

179 

179.00 

Error 

174 

6.62 

0.04 

Model  3 

2 

167.19 

Model  4 

5 

172.34 

Difference 

3 

5.18 

1.73 

43.25  .01 

Models  5  & 

6 

Total 

179 

179.00 

Error 

172 

5.73 

0.06 

Model  5 

4 

168.26 

Model  6 

7 

173.27 

Difference 

3 

5.01 

1.67 

50.15  .01 

Models  7  &  8 

Total 

179 

179.00 

Error 

171 

4.65 

0.03 

Model  7 

5 

173.45 

Model  8 

8 

174.35 

Difference 

3 

.90 

.30 

11.03  .01 
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Table  5 


Comparison  Among  Three  Models:  Compensatory  w/Unit  Weights,  Compensatory 


w/Differential  Weights.  &  Nonlinear  w/  Penalty 


Comparison 

Source 

DF 

SS 

MS 

F  fi  < 

Models  4  &  6 

Total 

179 

179.00 

Error 

172 

5.73 

0.03 

Model  4 

5 

172.38 

Model  6 

7 

173.27 

Difference 

2 

0.89 

.45 

13.39  .01 

Models  4  i  8 

Total 

179 

179.00 

Error 

171 

4.65 

0.05 

Model  4 

5 

172.38 

Model  8 

8 

174.35 

Difference 

3 

1.97 

.66 

24.01  .01 

Models  6  &  8 

^tal 

179 

179.00 

Error 

171 

4.65 

0.03 

Model  6 

7 

173.27 

Model  8 

8 

174.35 

Difference 

3 

1.08 

1.08 

39.71  .01 

7/ 

✓ 
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Table  6 

Cluster  Analysis  Results 


Mean  Regression  Estimates 


Cluster 

N 

CTP 

ELS 

MPO 

Intercept 

Cluster 

1 

26 

.38 

.39 

.38 

-.56 

Cluster 

2 

144 

.35 

.30 

.34 

.05 

Cluster 

3 

9  .23 

.23 

.30 

.74 

Figure  1.  Regression  Weights  by  Cluster 


-3 
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Characteristics  of  judges  within  cluster.  Chi -squared  analyses 
were  conducted  to  determine  if  membership  in  five  judge  categories 
(rank,  command,.  MOS,  gender,  and  race)  and  membership  in  the  three 
clusters  were  independent.  The  results  can  be  found  in  Table  7.  The 
values  for  rank,  command,  and  MOS  were  significant,  suggesting 
that  membership  in  these  groups  is  not  independent  from  membership  in 
the  three  clusters.  Gender  and  race  membership  appear  to  be 
independent  from  membership  in  the  clusters. 


Table  7 

Chi -Squared  Analysis  of  Rater  Characteristics  and  Cluster  Membership 


Rater  Characteristic 

N 

Strict 

Cluster 

Moderate 

1  Cluster  2 

Lenient 

Cluster  3 

X2 

Rank 

NCO 

93 

18 

73 

2 

27.80 

.01 

Officer 

74 

2 

65 

7 

Civilian 

11 

6 

5 

0 

Command 

Training  Sites 

55 

18 

37 

0 

23.82 

.01 

Field  Site 

124 

8 

107 

9 

MOS 

IIB 

81 

22 

59 

0 

37.56 

.01 

63B 

46 

0 

38 

8 

71L 

52 

4 

47 

1 

Gender 

Female 

22 

2 

18 

2 

1.34 

.51 

Male 

156 

24 

125 

7 

Race 

Black 

45 

6 

38 

1 

10.42 

.24 

American  Indian 

1 

1 

0 

0 

Hispanic 

15 

2 

13 

0 

White 

104 

14 

82 

8 

Other 

13 

3 

10 

0 

140 


I 


Discussion 


The  results  of  the  present  research  support  several  conclusions: 
First,  that  the  average  judgments  are  highly  linear.  This  finding  is 
consistent  with  the  review  of  information  processing  in  judgment 
conducted  by  Slovic  and  Lichtenstein  (1971),  as  well  as  the  conjoint 
measurement  analysis  conducted  by  Sadacca,  Park,  and  White  (1986)  to 
investigate  the  weighting  of  performance  constructs. 

Second,  although  the  linear  model  fits  very  well,  a  small  but 
statistically  significant  weight  was  found  for  the  inclusion  of  the 
standard  deviation  variable  into  the  regression  equation;  and  the 
for  the  model  with  the  standard  deviation  was  significantly  larger 
than  the  for  the  model  without  the  standard  deviation.  This 
finding  shows  a  small,  but  significant  nonlinearity  in  the  judges' 
judgment  policies.  Two  explanations  can  be  made  for  this  nonlinear 
finding:  (1)  judges  are  overly  influenced  by  extreme  values  on  one  of 
the  three  dimensions  (i.e.,  the  largest  standard  deviations  are  found 
when  two  of  the  dimensions  have  a  high  value  and  the  remaining 
dimension  has  a  low).  This  explanation  is  supported  by  the  research 
conducted  by  Fiske  (1980)  who  found  that  extreme  information  was  more 
salient  in  the  overall  judgment,  (2)  judges  may  have  penalized  the 
hypothetical  soldiers  for  inconsistent  behavior.  Therefore,  greater 
inconsistencies  among  the  performance  dimensions,  i.e.,  larger 
standard  deviations  resulted  in  larger  penalties. 

The  third  conclusion  that  is  supported  by  the  results,  concerns 
the  weighting  of  the  three  dimensions.  It  appears  that  judges  tend 
to  weight  each  of  the  three  dimensions  equally.  This  finding 
contradicts  previous  research  by  Dawes  and  Corrigan  (1979)  and  Slovic 
and  Lichtenstein  (1971)  who  found  that,  for  somewhat  more  complex 
stimuli,  the  overall  judgments  were  often  driven  by  information  on  a 
single  dimension.  Project  A  research  (Sadacca,  et  al .  1986)  while 
using  five  performance  dimensions  (the  three  used  in  the  current 
research  plus  General  Soldiering  Proficiency  (GSP),  and  Physical 
Fitness  and  Military  Bearing  (PFB)  found  significant  differences 
among  the  dimensions. 

The  fourth  conclusion  that  can  be  drawn  from  the  results 
concerns  the  scaling  of  the  stimulus  items.  Table  5  shows  that  the 
R^s  for  the  rescaled  models  are  higher  than  the  simple  integer 
scaling  in  all  cases  except  the  multiple  hurdles  model.  This  finding 
suggests  that  the  conjoint  measurement  approach  should  always  be 
accompanied  with  a  rescaling  of  the  stimulus  values  unless  a  multiple 
hurdles  model  is  assumed.  This  also  shows  that  the  psychological 
distance  between  the  performance  standards  is  not  necessarily  the 
same. 


The  final  conclusion  is  that  judges  differ  in  their  leniency, 
and  leniency  is  related  to  the  job,  command,  and  rank  of  the  judge. 
The  results  suggest  that  overall  standards  should  be  set  within  a 
job.  Also,  the  largest  proportion  of  moderate  judges  comes  from  the 
Officers  and  the  field  (FORSCOM)  sites. 
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CHAPTER  10: 


CONCLUSIONS  AND  RECOMMENDATIONS 


Synthetic  Val idation 

At  the  completion  of  Phase  I  of  the  research  project,  we  have 
shown  that  synthetic  validation  can  be  successfully  conducted. 
Chapters  3  and  4  documented  that  Army  SMEs  were  able  to  use  the  three 
job  component  models  to  describe  the  content  of  the  jobs.  We  also 
found  that  SMEs  were  consistent  in  identifying  the  critical 
components  of  their  jobs.  We  also  found  that  the  types  of  judges 
(i.e.  of  different  rank  and/or  assignment)  yielded  highly  similar  job 
descriptions.  In  using  the  Task  and  Activity  job  component  models  to 
describe  jobs,  SMEs  reported  that  the  instruments  provided  adequate 
coverage  of  the  job  content. 

Results  in  Chapter  5  provided  additional  support  that  testing 
and  measurement  experts  can  reliably  and  accurately  estimate 
validities  of  predictor  measures  for  job  performance.  We  found  that 
the  judged  validities  were  very  similar  to  empirical  validities  from 
Project  A.  There  was  some  indication  that  the  judged  validities 
underestimated  empirical  validities  that  have  been  corrected  for 
range  restriction. 

Chapter  6  showed  that  there  is  more  than  one  way  to  synthesize 
the  component  validities  to  obtain  a  prediction  composite  for  each 
job.  The  job  component  models,  in  combination  with  various  weighting 
schemes,  yielded  validities  with  different  properties.  For  example, 
the  highest  absolute  validities  were  obtained  by  the  Task  Category 
Model,  whereas  the  highest  differential  prediction  was  offered  by  the 
Job  Activity  Model. 

Results  reported  in  Chapter  7  showed  that  judges  who  are  more 
knowledgeable  about  their  jobs  provided  more  accurate  judgments.  We 
will  continue  to  obtain  additional  data  in  Phase  II  to  confirm  or 
disconfirm  that  finding. 

While  there  are  additional  research  issues  to  contend  with,  the 
Phase  I  results  are  very  positive.  Job  performance  in  the  each  of 
the  three  Phase  I  MOS  can  be  predicted  accurately  with  predictor 
batteries  determined  by  synthetic  procedures. 

As  indicated  in  the  Introduction,  these  Phase  I  results  were 
reviewed  by  the  Scientific  Advisory  Committee.  A  number  of  issues 
were  raised  by  the  SAC.  In  the  next  section,  we  will  address  how 
each  of  these  issues  affects  our  plans  for  data  collection  and 
analysis  during  Phase  II  of  the  project.  These  issues  include: 
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t  the  nature  of  the  job  description  ratings, 

•  the  level  of  specificity  for  the  job  component  models, 

•  the  type  of  raters  to  be  used, 

•  the  details  of  the  attribute  model, 

•  optimal  procedures  for  development  of  predictor  composites, 
and 

•  replication  of  Phase  I  results  to  additional  jobs. 

Job  description  ratings.  During  Phase  I,  we  asked  SMEs  to  rate 
each  Task  and  Activity  job  component  in  terms  of  the  frequency  of 
performance,  the  importance  for  Core  Technical,  General  Soldiering, 
and  overall  performance.  As  noted  earlier,  the  importance  ratings 
for  Core  Technical  Proficiency  were  used  in  weighting  the 
synthetically  derived  composites.  Two  suggestions  were  made  by  the 
scientific  advisors  that  warrant  further  investigation.  The  first 
suggestion  was  to  redefine  criticality  by  combining  frequency  and 
importance.  There  is  a  theoretical  rationale  for  forming  a  product 
of  frequency  and  importance  as  a  measure  of  each  component's 
contribution  to  overall  job  performance. 

A  second  suggestion  was  to  revisit  the  use  of  difficulty 
information.  A  limited  investigation  of  difficulty  ratings  was 
conducted  during  the  Pilot  Test.  One  concern  with  obtaining 
difficulty  information  was  that  it  may  depend  on  the  individual 
soldier--what  is  difficult  for  one  soldier  may  be  trivial  for 
another.  Asking  SMEs  to  focus  on  average  training  difficulty  might 
be  meaningful.  In  other  words,  they  might  consider  the  duration  and 
resources  for  training  enlistees  on  a  particular  component  of 
performance. 

A  related  suggestion  was  to  investigate  the  importance  of 
particularly  good  performance  (or  the  cost  of  a  particularly  bad 
performance),  rather  than  just  asking  for  the  importance  of  a  task  or 
an  activity  in  general.  Then  more  weight  would  be  given  to 
predictors  of  job  component  performance  where  the  difference  between 
good  and  bad  performance  matters  the  most  (and  where  job  incumbents 
do  vary  in  the  level  of  their  performance). 

As  a  result  of  these  suggestions,  we  will  examine  alternative 
ways  of  rating  the  criticality  of  job  components  and  of  using  this 
information  to  obtain  weights  for  the  different  job  component 
prediction  equations. 

Level  of  specificity  in  the  .lob  component  models.  There  were 
two  questions  that  the  SAC  raised  concerning  the  relationship  among 
instrument  length  and  reliability  and  discriminabil ity.  The  first 
issue  centers  on  the  degree  to  which  instrument  length  (i.e.,  the 
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number  of  components)  affected  the  reliability  of  the  component 
ratings  and  the  validity  of  the  resulting  prediction  equations.  One 
suggestion  was  that  differences  between  the  task  and  activity  model 
reliabilities  may  be  partly  due  to  differences  in  the  number  of  items 
(job  components)  in  these  two  models.  This  argument  suggests  that 
increasing  the  number  of  items  would  lead  to  greater  job  description 
reliability. 

The  second  concern  was  that  the  high  correlation  among  profiles 
generated  by  the  task  model,  in  comparison  to  the  activity  model,  may 
be  an  artifact  because  of  the  greater  number  of  categories  in  the 
task  model.  Any  two  jobs  sharing  a  larger  number  of  categories  rated 
not  relevant  can  result  in  high  profile  correlations  and  perhaps 
also  a  reduction  in  discriminant  validity. 

However,  improving  discriminant  validity  through  reducing  the 
number  of  categories  might  threaten  instrument  reliability.  Given 
t.''. is  tradeoff,  some  systematic  investigation  of  the  effects  of 
varying  the  number  of  categories  in  the  task  and  activities  models  is 
warranted  and  will  be  addressed  in  Phase  II. 

Types  of  raters.  In  general,  the  mean  job  description  profiles 
generated  by  different  groups  of  raters  were  highly  correlated. 
Overall,  there  was  somewhat  less  agreement  for  NCO  ratings  in 
comparison  to  officer  ratings.  The  lower  agreement  among  NCOs  may 
reflect  their  proximity  to  the  job.  NCOs  may  be  more  familiar  with 
specific  variation  in  job  duties  for  an  MOS,  while  Officers  may  focus 
more  on  the  "stereotypic"  elements  of  the  job  that  are  common  to 
different  duty  areas. 

The  larger  differences  between  NCOs  and  officers  for  the  63B 
(mechanic)  job  descriptions  was  a  cause  for  some  concern.  There  may 
well  be  greater  agreement  for  some  types  of  jobs  than  for  others  and, 
so  far,  we  have  only  examined  three  different  jobs.  Continued 
investigation  of  differences  between  different  types  of  raters  for 
the  seven  additional  Phase  II  jobs  is  warranted. 

Attribute  model.  A  number  of  suggestions  were  made  concerning 
possible  changes  to  the  attribute  model.  These  included: 

•  drop  the  attribute  "closure" 

•  drop  interests  or  obtain  validities  against  overall  job 
performance  only 

•  instead  of  focusing  on  component  validities,  obtain 
validities  for  the  five  Project  A  performance  constructs  of 
Core  Technical,  General  Soldiering,  etc. 

•  if  obtaining  component  validities  was  essential,  those 
judgments  should  be  made  on  an  abbreviated  list  of  tasks 
and/or  activities 
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Further  investigation  of  these  options  will  be  pursued  during  Phase 
II. 


Development  of  predictor  composites.  Phase  I  analyses  showed 
that  different  approaches  to  developing  attribute  and  component 
weights  led  to  noticeable  differences  in  absolute  and  discriminant 
validities.  Further  exploration  of  these  differences  across  a  wider 
range  of  jobs  in  Phase  II  will  be  needed.  Further  investigation  of 
the  Phase  I  data  will  suggest  re-scaling  of  the  attribute  ratings  and 
component  weights.  Phase  II  data  would  verify  the  feasibility  of 
such  re-scaling.  With  the  addition  of  different  types  of  job 
description  ratings  (difficulty  as  well  as  frequency  and  importance), 
a  number  of  exploratory  analyses  also  will  be  required  during  Phase 
II. 

Replication.  The  scientific  advisors  endorsed  the  importance  of 
using  the  Phase  II  data  collection  to  replicate  Phase  I  findings. 

The  Phase  I  results  are  impressive.  However,  until  more  evidence  of 
generalization  to  more  jobs,  we  have  to  be  cautious  about  the 
results.  In  Phase  II,  we  will  investigate  generalization  to  seven 
additional  jobs  bringing  the  total  sample  of  jobs  up  to  ten. 

Standard  Setting 

There  are  three  major  issues  to  be  addressed  by  the  standard 
setting  research:  (1)  setting  standards  for  individual  performance 
dimensions;  (2)  setting  overall  performance  standards;  and  (3) 
linking  selection  test  standards  to  overall  performance  expectancies. 
The  first  two  issues  were  addressed  in  Phase  I  and  are  also  included 
in  the  plans  for  Phase  II;  the  third  issue  will  be  addressed  later  in 
the  project. 

Phase  I  progress  in  investigating  standard  setting  methodology 
includes: 

•  defining  four  performance  levels  (unacceptable,  marginal, 
acceptable,  and  outstanding), 

e  designing  three  standard  setting  approaches  for  obtaining 
component  level  standard  (these  approaches  are  soldier- 
based,  task-based,  and  critical  incident-based), 

•  designing  one  method  for  obtaining  overall  standard 

•  developing  the  initial  judgment  protocols  for  the  four 
exercises,  and 

•  testing  the  protocols  on  three  MOS. 

As  planned,  the  standard  setting  research  in  Phase  I  was  not  as 
well -developed  as  the  synthetic  validation  work.  Field  testing  of 
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standard  setting  procedures  did  not  occur  until  Phase  I.  Thus,  the 
Phase  I  data  collection  on  standard  setting  may  be  viewed  as  a  pilot 
effort.  However,  the  results  of  this  pilot  effort  were  informative 
for  future  endeavors. 

For  the  definition  of  the  four  performance  standards,  we  found 
that  SMEs  had  no  difficulty  in  understanding  and  in  working  with 
these  definitions.  In  Chapter  8,  we  reported  that  SMEs  found  that 
the  outcomes  of  the  performance  standards  were  meaningful  and  not 
improbable. 

Phase  I  results  documented  in  Chapter  8  also  indicated  that  SMEs 
experienced  some  difficulties  with  the  task-based  descriptions.  In 
particular,  judges  were  confused  by  descriptions  of  the  probability 
of  passing  particular  steps  that  were  scored  go  or  no-go.  Some 
judges  wanted  more  step-level  information  and  others  felt  that  step- 
level  information  did  not  help.  Judgments  based  on  the  task 
description  resulted  in  stricter  standards  in  comparison  with  the 
other  two  approaches.  Stricter  standards  meant  that  there  was  a 
higher  proportion  of  unacceptable  performance  among  incumbents.  Few 
significant  difference  in  levels  of  agreement  were  found  between 
methods  or  between  normative  and  practice  conditions. 

On  the  overall  judgment  exercise  reported  in  Chapter  9,  results 
indicated  judges'  strategies  could  be  described  very  well  with  a 
linear  (compensatory)  model.  Nonetheless,  statistically  significant 
improvements  in  the  model  were  achieved  by  differential  spacing  of 
the  different  levels  of  performance  and  by  adding  a  "penalty"  for 
inconsistent  performance  across  different  dimensions. 

As  with  the  synthetic  validation  research,  the  Phase  I  standard 
setting  results  pointed  to  areas  for  further  investigation  for  Phase 
II  of  this  project.  These  areas  are: 

t  clarification  of  acceptable  performance 

t  simplification  of  task-based  rating  protocol 

•  generalization  across  jobs 

§  combining  multiple  standards 

Clarification  of  acceptable  performance.  The  scientific 
advisors  expressed  concern  that  the  definition  of  "acceptable"  may 
not  be  positive  enough  and  if  it  were  consistent  with  the  Army's 
position.  They  were  assured  that  the  Army  does  encourage  the 
"acceptable"  soldiers  to  re-enlist.  However,  owing  to  the  limited 
resources,  the  extra  effort  should  be  directed  at  encouraging 
"outstanding"  soldiers  to  re-enlist. 

Simplification  of  the  task-based  rating  protocols.  At  the 
Phase  I  standard  setting  workshops,  it  was  evident  that  the  task- 
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based  rating  protocols  could  be  improved.  One  suggestion  was  to 
provide  less  detailed  information.  For  example,  it  might  be 
sufficient  to  list  the  tasks  for  a  performance  dimension  and  to  ask 
SMEs  to  judge  the  percent  of  steps  in  these  tasks  soldiers  would  have 
to  pass  to  meet  a  particular  standard.  This  suggestion  reflects  the 
view  that  SMEs  are  familiar  with  the  tasks  and  their  critical  steps, 
and  the  Soldiers'  Manuals  contain  sufficient  information  for  such 
judgments. 

An  opposing  point  of  view  was  that  more  information  might  yield 
more  accurate  judgments.  In  fact,  several  of  the  Phase  I  judges 
requested  more  information.  One  approach  that  would  both  provide 
more  and  also  clearer  information  would  be  to  provide  complete  hands- 
on  score  sheets  for  a  carefully  constructed  group  of  hypothetical 
examinees.  A  question  that  requires  careful  monitoring  is  whether 
judges  would  get  bogged  down  evaluating  tradeoffs  between  tasks  and 
task-steps  and  would  not  complete  the  ratings  of  these  hypothetical 
soldiers  reliably. 

There  are  a  number  of  possible  approaches  to  generating  more 
detailed  information  about  task  performance  levels.  First,  we  could 
sample  from  the  data  provided  by  Project  A  soldiers.  This  approach 
would  allow  us  to  build  on  other  information  that  is  known  about 
these  soldiers,  but  might  require  ratings  on  relatively  large  samples 
of  soldiers  to  compensate  for  detailed  individual  differences.  An 
alternative  approach  would  be  to  generate  hypothetical  data,  so  that 
the  task-step  scores  formed  a  perfect  Guttman  scale  (where  a 
hypothetical  soldier  would  pass  a  particular  step  only  if  he  or  she 
also  passed  all  easier  steps). 

Given  opposing  suggestions  for  modifications  to  the  task-based 
standard  setting  instruments,  it  is  reasonable  to  introduce  planned 
variations  in  the  level  of  task  and  step  information  provided  during 
Phase  II.  This  will  allow  us  to  test  whether  better  agreement  and 
more  or  less  lenient  standards  result  from  more  or  less  detailed 
information. 

Generalization  across  jobs.  The  three  jobs  analyzed  in  Phase  I 
were  quite  different  and  did  not  share  many  common  components  in  the 
core  technical  area.  An  issue  to  be  investigated  in  Phase  II  is 
whether  standard  setting  exercises  can  be  generated  for  specific 
components  or  dimensions  that  generalize  across  jobs.  This  question 
is  particularly  important  for  the  incident-based  approach  because 
job-specific  incidents  were  only  collected  for  nine  jobs  in  Project 
A.  Specific  critical  incidents  are  available  for  only  three  of  the 
seven  Phase  II  jobs.  We  will  have  to  "borrow"  incidents  from  related 
jobs  or  jcb  components  in  creating  incident-based  instruments  for  the 
other  four  Phase  II  jobs. 

Combining  multiple  standards.  In  Phase  I,  we  examined  the  way 
in  which  judges  rated  the  overall  performance  of  soldiers  when 
provided  with  information  about  performance  on  three  general 


dimensions.  In  Phase  II,  we  will  focus  on  combining  standards  for 
multiple  dimensions  within  Core  Technical  Proficiency.  Planned 
variation  in  the  number  of  different  dimensions  rated  by  each  judge 
will  be  introduced. 


Replication.  As  with  the  synthetic  validation  research, 
replication  of  Phase  I  results  for  a  larger  number  of  jobs  will  be  a 
primary  objective  for  Phase  II.  Nonsignificant  differences  between 
rater  types  and  methods  may  or  may  not  be  replicated  for  additional 
samples. 


Steps  for  Further  Research 


The  iterative  design  of  the  Army  Synthetic  Validation  Project 
will  allow  us  to  address  the  issues  raised  above  during  Phases  II  and 
III  of  the  project.  Phase  II  of  the  project  is  scheduled  to  run  from 
November  1988  through  June  1989,  with  data  collection  workshops 
scheduled  for  January  through  March  1989.  Seven  different  MOS  will 
be  examined  during  Phase  II. 


During  Phase  II,  further  exploratory  variation  in  procedures 
will  be  examined.  For  synthetic  validation,  we  will  examine  the  use 
of  different  levels  of  specificity  in  the  job  component  models  and  of 
different  ratings  for  deriving  the  job  component  weights.  In  the 
standard  setting  area,  we  also  will  examine  different  levels  of 
specificity  in  the  performance  domains  for  which  standards  will  be 
set  and  also  will  examine  the  general i lability  of  particular 
standards  across  different  jobs. 


Current  plans  call  for  Phase  III  to  be  used  primarily  to  cross- 
validate  exploratory  findings  from  Phase  I  and  Phase  II.  Phase  III 
will  include  13  additional  jobs,  some  of  which  will  be  outside  of  the 
Project  A  sample.  Phase  III  will  also  provide  a  final  opportunity  to 
test  hypotheses  concerning  optimal  procedures  for  selecting  and 
training  the  judges. 


APPENDIX  A 

Minutes  of  Scientific  Advisory  Committee  Meeting 

Notes  From  the  Spthetic  Validity  Project 
Scientific  Advisory  Committee  Meeting 
held  on  November  9  and  10, 1988 
in  Washington,  DC 


Attendees:  Phil  Bobko,  Bob  Guion,  Dick  Jaeger,  Bob  Liim,  and  Joyce  Shields  of  the  SAC; 
project  advisor  Ron  Hambleton;  Jane  Arabian  and  Larry  Hanser  of  ARI;  Wei  Jing  Chia, 
Phil  Szenas,  and  Laurie  Wise  of  AIR;  Cyndi  Owens-Kurtz  and  Norm  Peterson  of  PDRI; 
and  John  Campbell  and  Gene  Hoffman  of  HumRPO. 


Day  1:  Job  Component  Models.  Expert  Judgment  Research, 
and  Synthetic  Equations 

Update  and  Overview  of  Meeting 

Jane  Arabian  welcomed  everyone  to  the  meeting  and  reviewed  plans  for  the  meeting. 
The  first  day  of  the  meeting  would  be  devoted  to  review  of  analyses  of  data  gathered 
for  the  job  component  models,  expert  validity  estimation,  and  synthetic  validation.  The 
group  would  spend  the  next  half  day  reviewing  initial  standard  setting  results.  Jane  also 
asked  SAC  members  to  suggest  ways  of  consolidating  or  modifying  models  for  future 
testing  and  to  provide  options  and  directions  for  the  project  staff  for  the  remainder  of 
the  project. 

Laurie  Wise  described  progress  made  since  the  November  1987  SAC  meeting.  He  listed 
the  Pilot  Test  of  job  component  models,  the  Phase  I  data  collection,  and  the  expert 
validity  data  collection,  as  major  project  accomplishments  during  the  course  of  the  last 
year.  Laurie  also  pointed  out  that  there  was  a  considerable  list  of  results  for  the  SAC 
to  review. 
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Major  activities  for  the  coming  year  will  include  revising  models  and  procedures  for 
Phase  II  data  collection  in  January-March  1989.  In  Phase  11, 7  more  Project  A  jobs  will 
be  studied.  Another  SAC  meeting  is  planned  for  next  July  or  August. 


Task  and  Activity  Models:  Pilot  and  Phase  I  Test  Results 

Wei  Jing  Chia  presented  results  of  the  Pilot  and  Phase  I  Test  results.  In  both  field 
tests,  three  MOS  (Infantryman,  Vehicle  Mechanic,  and  Administrative  Specialist)  were 
studied.  The  results  indicated  that  both  the  Task  and  Activity  job  component  models 
produced  reliable  Job  descriptions  from  Army  SMEs.  There  was  adequate  discriminability 
among  jobs  using  importance  ratings  for  Core  Technical  Proficiency.  Although  Officers 
as  a  group  were  more  consistent  in  their  ratings  than  NCOs,  the  groups  shared  very 
similar  ordering  of  the  components  in  terms  of  the  importance  ratings.  Based  on 
evaluation  of  the  two  job  component  instruments  provided  by  the  SMEs,  about  70%  of  the 
key  duties  for  the  three  jobs  were  contained  in  the  instruments.  When  asked  to  supply 
components  that  were  missing,  these  SMEs  were  unable  to  do  so  or  identity  components 
that  were  clearly  contained  in  the  existing  instruments.  Norm  Peterson  stated  that  the 
70%  level  of  coverage  is  about  what  is  usually  found  when  judges  are  asked  to  compare 
job  component  instruments. 

A  number  of  issues  were  raised  during  the  presentation.  Phil  Bobko  suggested  a 
multiplicative  model  that  weighted  job  components  by  the  product  of  importance  and 
frequency  of  performance  might  provide  better  discrimination.  Phil  and  Joyce  Shields 
also  suggested  including  some  measure  of  difficulty  in  terms  of  time  to  train  into 
measuring  criticality  of  a  job  component.  Phil  stated  that  it  is  important  to  take  the 
"individual"  out  of  the  difficulty  ratings  (e.g.,  what  is  difficult  for  one  soldier  may  not 
be  for  another). 

Ron  Hambleton,  Dick  Jaeger,  and  Norm  recommended  that  in  computing  single  rater 
reliability,  the  variance  due  to  MOS  should  be  considered  error  variance.  Ron  asked 
about  how  the  role  of  instrument  length  might  influence  the  reliabilities.  Phil  responded 
that  it  depends  on  the  number  of  "Not  Part  of  Job"  responses.  Laurie  added  that  the 
"Not  Part  of  Job"  also  plays  an  important  role  in  the  instrument’s  ability  to  discriminate 
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among  jobs.  Dick  concluded  by  saying  that  "the  instrument  is  the  instrument"  and  should 
be  evaluated  more-or-less  "as  is". 

The  next  discussion  concerned  the  cause  and  implication  of  the  lower  reliabilities  for  the 
NCOS  as  compared  to  the  Officers.  Laurie  stated  that  the  lower  NCO  reliabilities  may  be 
due  to  the  fact  that  these  NCO  entered  the  Army  during  the  period  of  the  mis-norming 
of  the  ASVAB  that  occurred  in  the  late  1970’s  and  early  1980’s.  Norm  suggested  that 
Officers  may  share  a  common  stereotype  that  would  tend  to  inflate  the  reliabilities 
suggesting  that  the  NCOs  more  accurately  reflect  the  real  job.  John  Campbell  pointed 
out  that  the  stereotype  might  be  the  correct  (organizational  policy)  view  of  the  job. 

Jane  stated  that  maybe  we  do  not  want  the  reliabilities  to  be  too  high  because  we  want 
to  sample  SMEs  that  have  a  wide  range  of  duty  experience.  Phil  suggested  that  the 
NCO-Officer  distinction  may  make  more  of  a  difference  for  some  MOS  than  for  others. 
Jane  said  that  we  were  taking  this  into  account  by  using  Warrant  Officers  as  SMEs  for 
some  MOS  because  they  are  more  involved  with  the  direct  supervision  of  soldiers 
performing  job  duties.  The  lower  co-officer  agreement  for  63B  warrants  further 
investigation. 


Attribute  Model:  Pilot  and  Phase  I  Test  Results  fSoldier  Method’! 

Cyndi  Owens-Kurtz  presented  results  of  Army  SME  ratings  and  rankings  of  the  validity 
of  attributes  for  predicting  job  performance.  Despite  the  apparent  complexity  of  the 
task,  the  results  indicated  that  the  SMEs  understood  the  materials  and  provided  useful 
data.  Inter-rater  reliabilities  were  acceptable  and  indicated  that  15  raters  should  supply 
sufficiently  reliable  results  in  most  cases.  Again,  she  reported  that  Officers  have  better 
inter-rater  agreement  than  NCOs  do.  The  pattern  of  judged  validities  of  attributes  for 
job  performance  corresponded  to  the  pattern  found  in  Project  A  research.  There  was 
validity  differentiation  across  jobs  for  Core  Technical  Proficiency,  but  validity 
generalization  of  the  attributes  for  General  Soldiering,  Effort  and  Leadership,  Personal 
Discipline,  and  Physical  Fitness  and  Military  Bearing. 

Dick  raised  a  concern  about  the  readability  of  the  attribute  definitions  by  SMEs.  He 
speculated  that  the  complexity  of  the  definitions  might  steer  the  SMEs  to  select  the 
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middle  point  of  the  rating  scale.  Cyndi  disagreed  and  pointed  out  that  the  pattern  of 
validities  of  the  Army  SMEs  corresponded  well  to  that  of  the  psychologists. 


Judge  Screening:  Rater  Characteristics  and  Rating  Fidelity 

Phil  Szenas  presented  a  causal  model  of  individual  difference  variables  on  rating 
accuracy.  Individual  difference  variables  included  general  aptitude,  job  experience,  years 
of  education,  knowledge  of  own  job,  and  knowledge  of  job  rated.  Rating  accuracy  was 
defined  as  the  correlation  of  individual  ratings  with  the  mean  ratings  of  the  target 
group.  He  found  that  the  same  path  model  accounted  for  the  rating  accuracy  of  the 
Task,  Activity,  and  Attributes  rating  accuracy.  Knowledge  of  own  job  and  knowledge  of 
job  rated  were  key  determinants  of  rating  accuracy.  Therefore,  he  suggested  that  judge 
selection  might  be  predicated  on  job  knowledge. 

Although  the  analysis  was  exploratory,  the  results  drew  a  lot  of  interest.  General 
aptitude  had  a  relatively  minor  role  on  rating  accuracy  apart  from  its  relationship  to  job 
knowledge.  According  to  Bob  Guion,  general  aptitude  should  still  serve  as  a  good  screen 
even  though  its  influence  on  rating  accuracy  was  indirect.  Larry  Hanser  suggested 
modifying  the  model  by  keeping  only  the  direct  effects  of  general  aptitude  on  rating 
accuracy.  John  also  pointed  out  that  the  causal  order  of  the  individual  difference 
variables  may  not  be  as  critical  if  judge  selection  is  based  on  all  those  variables.  Bob 
Linn  indicated  that  the  pattern  of  correlations  of  knowledge  of  own  job  with  fidelity 
coefficients  of  the  three  models  suggested  different  paths.  Despite  limited  data,  Laurie 
stated  that  the  results  indicated  a  simple  minded  conclusion:  a  good  judge  is  one  who  is 
knowledgeable  of  the  job. 


Summary  of  the  Job  Component  Models 

Laurie  provided  a  summary  of  the  three  presentations  given  in  the  morning.  Reliabilities 
found  for  the  three  job  component  models  were  fairly  consistent.  The  resulting  job 
descriptions  and  attribute  patterns  for  the  different  jobs  made  sense.  There  was  some 
indication  that  officers  were  somewhat  more  reliable  than  NCOs.  Some  differences  were 
also  observed  between  FORSCOM  and  DOTD  judges.  DOTD  judges  were  more  reliable 
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with  Activities  job  description.  Laurie  suggested  that  perhaps  they  were  more  familiar 
with  dealing  with  components  at  that  level  of  abstraction.  Overall,  there  was  no 
significant  differences  between  descriptions  of  the  groups  of  Judges  as  evident  from  the 
high  correlations  of  the  mean  rating  profiles.  Therefore,  the  job  component  models  have 
passed  the  first  hurdle  in  providing  reliable  job  descriptions.  More  work  will  be 
undertaken  to  see  if  frequency  information  leads  to  higher  validities  and  discriminability. 

As  the  three  job  component  models  yielded  comparable  data,  a  decision  to  drop  one  or 
two  models  will  not  be  easy.  Norm  argued  that  the  choice  should  be  based  on  the 
synthetic  validities.  Ron  suggested  data  collection  effort  might  be  another  criterion. 
Laurie  agreed  that  we  should  monitor  data  collection  time  requirements  more  closely  in 
Phase  II.  Norm,  however  felt  that  the  half-day  to  gather  data  for  all  three  models  does 
not  present  an  operational  problem. 

Norm  added  that  the  end  of  Phase  I  marks  a  major  accomplishment  for  the  project.  We 
now  have  a  set  of  procedures  for  deriving  synthetic  equations  which  will  predict 
performance  on  a  small  set  of  jobs.  Larry  raised  a  concern  whether  these  research 
procedures  will  also  work  under  operational  conditions. 

Phil  Bobko  questioned  the  application  of  synthetic  validation  procedures  for  a  job  that  is 
being  created.  Joyce  suggested  that  it  might  be  worthwhile  to  include  new  jobs  that  are 
being  planned,  e.g.  IIB  Bradley  and  compare  job  descriptions  from  the  judges  who  are 
knowledgeable  and  those  who  are  not  knowledgeable  about  the  vehicle.  Laurie  pointed 
out  that  the  iterative  approach  of  the  project  can  allow  us  to  address  that  issue.  Not 
much  can  be  done  about  the  Phase  II  design  as  troop  support  requests  have  been  filed, 
but  in  Phase  III,  we  will  attempt  to  study  new  jobs  and/or  jobs  that  are  undergoing 
significant  changes. 

Gene  raised  the  issue  of  focussing  on  average  importance  ratings  versus  obtaining  a 
product  as  a  result  of  group  effort.  Laurie  pointed  out  that  the  single  rater  reliabilities 
have  remained  high  from  field  test  to  field  test.  John  speculated  what  difference  might 
be  observed  in  the  ratings  obtained  by  averaging  across  10  judges,  by  a  delphi  procedure 
of  10  judges,  or  by  10  highly  selected  judges.  Norm  did  not  like  the  delphi  approach 
because  of  time  and  cost  considerations.  Bob  Guion  didn’t  think  the  group  problem 
solving  research  would  necessarily  generalize  to  a  task  like  job  description. 

I 
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Expert  Judgment  of  Validities  (psychologist  method^ 


Norm  outlined  the  "psychologist"  method  for  obtaining  predictor-job  component  validities. 
The  sample  of  69  judges  included  a  large  number  of  personnel  psychologists  with 
excellent  credentials.  He  also  described  his  rationale  for  forming  subgroups  of  experts 
on  the  basis  of  psychological  expertise  and  military  familiarity.  There  was  acceptable 
levels  of  inter-rater  reliabilities  for  the  group  of  judges  as  a  whole  as  well  as  by  MOS, 
and  by  instrument.  There  was  little  support  for  differences  in  reliabilities  by  grouping 
according  to  expertise  and  military  familiarity. 

The  pattern  of  judged  validities  for  the  three  jobs  made  sense  and  was  as  expected.  For 
example,  Mechanical  Comprehension  received  a  higher  validity  for  63B  than  either  of  the 
other  jobs.  Mean  validities  for  core  technical  performance  were  highest  for  cognitive 
abilities  and  lowest  for  interests;  mean  validities  for  psychomotor  and  physical  abilities 
had  intermediate  values.  The  psychologists’  mean  validities  show  excellent 
correspondence  to  those  by  soldiers.  The  psychologists’  validities  in  most  instances 
underestimated  the  "true"  validities. 

If  there  was  a  need  to  gather  expert  validity  judgments  again,  Norm  proposed  the 
following  changes: 

•  drop  closure 

•  drop  interests  or  obtain  validities  against  overall  job  performance  only 

•  instead  of  focussing  on  component  validities,  obtain  validities  for  the  five 
Project  A  performance  constructs  of  Core  Technical,  General  Soldiering,  etc. 

•  if  obtaining  component  validities  was  essential,  we  should  settle  on  an 
abbreviated  list  of  tasks  and/or  activities 

According  to  Norm,  one  judge  made  a  distinction  between  performance  and  proficiency. 
Bob  Guion  said  that  the  term  "job  performance"  connotes  a  more  global  description  of 
work  to  include  attendance.  Phil  Bobko  added  that  a  typical-maximal  performance 
continuum  might  describe  the  performance/proficiency  distinction. 
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Comparison  of  Empirical  and  Synthetic  Prediction  Equations 


Laurie  described  the  two  steps  for  forming  the  synthetic  equations.  The  first  step 
involves  weighting  attributes  to  form  a  performance  prediction  composite  for  each  job 
component.  Each  of  the  performance  component  prediction  composite  is  then 
standardized.  In  the  second  step,  each  performance  component  prediction  composite  is 
weighted  by  the  job  component  importance  weights  obtained  for  each  job.  The  second 
step  is  not  needed  for  the  attribute  model. 

Empirical  equations  using  Project  A  data  served  as  the  benchmark  for  the  synthetic 
equations.  Two  criteria  were  critical:  (a)  absolute  validities  by  job  and  (b)  the  degree  of 
differential  prediction  across  the  three  jobs  in  Phase  I.  Three  types  of  weights  were 
tried  for  weighting  attributes  and  they  were  validity,  regression,  and  unit  weights.  Two 
types  of  weights  were  tested  for  weighting  job  components,  and  they  were  total  and 
adjusted  weights.  All  synthetic  equations  yielded  useful  validities  and  synthetic  validities 
using  some  combination  of  weights  approached  the  empirical  validities.  Regression 
attribute  weights  produced  the  lowest  validities.  The  highest  validities  were  obtained 
from  unit  attribute  weights.  Adjusted  component  weights  produced  the  best  differential 
VC  .iity.  The  attribute  model  worked  better  with  psychologists  than  with  soldiers.  The 
Task  model  yielded  higher  absolute  validities  and  lower  differential  validities  compared  to 
the  Activity  Model. 

A  number  of  issues  were  raised  during  the  presentation.  Bob  Guion  suggested  restricting 
the  predictor  battery  to  the  five  best  predictors.  Norm  related  his  experience  in  the 
LOMA  study  where  he  was  successful  in  limiting  the  test  battery  to  five  predictors  for 
each  job  without  sacrificing  overall  predictability.  Norm  suggested  setting  a  maximum 
amount  of  reduction  in  that  can  be  tolerated,  and  then  use  backward  stepwise 
regression  to  remove  predictors  until  this  criterion  is  met.  John  noted  that  it  is 
important  to  examine  the  content  of  the  equations.  Upon  examining  the  weights  of  the 
synthetic  equations,  Bob  Linn  pointed  out  the  63B  and  71L  had  very  different  equations. 

Bob  Guion  also  questioned  if  the  small  amounts  of  improvement  in  differential  validity 
were  worthwhile.  Larry  pointed  out  that  the  small  gains  do  accrue  significantly  for  a 
large  organization  like  the  Army. 
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Synthetic  Validation  Summary  and  Plans 


John  led  the  discussion  of  the  project  progress  up  through  Phase  I  and  plans  for  Phase 
II  of  the  project.  He  reminded  us  that  the  overall  goal  was  to  identify  prediction 
equations  for  jobs  for  which  empirical  validation  is  not  possible.  Two  solutions  are 
investigated:  validity  generalization  (VG)  and  synthetic  validation.  Traditional  validation 
has  been  carried  out  on  a  prototypic  job  in  a  job  family.  The  efficacy  of  synthetic 
validation  for  generalizing  validity  to  the  other  jobs  in  the  family  would  be  compared  to 
generalizing  validity  by  VG  procedures  from  the  prototypic  job. 

To  obtain  generalized  validity  via  synthetic  validation,  a  number  of  key  issues  must  be 
addressed:  (a)  what  is  the  unit  of  analysis?  (b)  what  judgments  are  required;  (c)  who 
makes  the  judgments?  (d)  what  is  the  procedure  for  deriving  judgment-based  prediction 
equations?  and  (e)  what  is  the  evidence  for  empirical  verification? 

Some  of  the  issues  have  been  address  by  the  Phase  I  data.  Further  analyses  of  Phase  I 
data  should  focus  on: 

•  comparison  of  subgroups  on  accuracy  of  judged  validities  for  the  attribute 
model 

•  comparison  of  judged  validities  v.  empirical  validities 

•  conducting  sensitivity  analyses  on  differential  prediction  on  realistic  selection 
and  classification  system  using  five  or  six  predictors 

Phase  II  plans  should  be  primarily  directed  at  replication  and  extension  of  Phase  I 
findings.  Replication  of  the  same  procedures  with  new  MOS  should  be  conducted. 
Extensions  should  explore  comparison  of  different  levels  of  job  element  taxonomies.  A 
number  of  sensitivity  analyses  of  equation  discriminability  should  be  conducted  by  varying 
(a)  job  similarity;  (b)  number  of  predictors;  (c)  predictor  content;  (d)  predictor  weights; 
(e)  judge  type;  (f)  level  of  taxonomy;  and  (g)  types  of  criticality  judgments. 

A  number  of  issues  were  raised  by  SAC  members  during  the  discussion.  One  issue  had  to 
do  with  the  testing  procedure.  John’s  suggestion  of  testing  applicants  on  only  a  small 
set  of  predictor  measures  elicited  questions  from  Bob  Linn  and  Bob  Guion  about  the 
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number  of  predictors  to  administer.  Some  variant  of  "adaptive"  test  selection  was 
proposed  by  Larry.  Joyce  suggested  a  multiple  hurdles  approach  to  sequencing  tests. 

When  John  discussed  the  option  of  dropping  interests  from  further  study,  Joyce  wanted 
to  know  what  the  Army’s  position  was  with  regards  to  using  interests  in  selection. 

Larry  felt  that  Project  A  longitudinal  validation  on  interest  was  not  far  along  enough  to 
inform  a  policy  on  the  use  of  interests.  Jane  added  that  the  preliminary  findings  are 
that  interests  can  compensate  for  deficits  in  some  general  aptitude. 

Another  issue  which  drew  some  discussion  was  the  selection  of  SMEs  for  job  description. 
John  was  concerned  about  the  mix  of  judges  with  regards  to  their  experience  of  the  job. 
Ron  suggested  examining  the  specific  duties  the  judges  had  in  the  MOS.  Jane  agreed 
that  we  might  ask  judges  to  list  the  duty  positions  they  had  supervised  in  the  MOS. 

John  proposed  comparing  high  and  low  ability  judges  in  terms  of  their  reliability, 
discriminability,  and  fidelity,  Joyce  felt  that  judge  heterogeneity  depends  on  the  MOS. 
For  example,  there  were  more  varied  assignments  for  63B.  Jane  mentioned  that  too  high 
inter-rater  reliabilities  may  not  be  desirable  because  that  implies  capturing  a  limited 
perspective  of  the  MOS. 

Another  issue  discussed  was  the  level  of  specificity  in  job  description.  So  far,  job 
description  has  proceeded  at  one  level.  John  proposed  to  describe  jobs  at  a  more 
abstract  level.  An  attempt  at  a  higher  level  abstraction  might  be  the  results  of  a  factor 
analyses  of  the  job  elements  based  on  their  pattern  of  attribute  validities.  Norm  ran  the 
analyses  and  results  were  distributed  at  the  meeting.  The  results  revealed  six 
interpretable  factors  with  a  good  mix  of  tasks  and  activities  defining  each  factor. 

Joyce  Shields  reminded  the  group  of  the  issued  concerning  measuring  the  "difficulty"  of 
performing  job  components.  Bob  Guion  said  that  there  is  a  difference  between  how 
important  a  job  component  is  for  a  job  and  how  important  it  is  to  do  the  job  component 
well.  Laurie  said  that  if  number  of  component  in  the  questionnaire  can  be  reduced,  that 
"importance,"  "frequency,"  and  "difficulty"  judgments  can  each  be  collected. 
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Day  2:  Standard  Setting  Results  and  Phase  II  Plans 


Overview  of  Standard  Setting  Research 

Laurie  described  the  three  major  issue  to  be  address  by  the  standard  setting  research: 

(1)  setting  standards  for  individual  performance  dimensions;  (2)  setting  overall 
performance  standards;  and  (3)  linking  selection  test  standards  to  overall  performance 
expectancies.  The  first  two  issues  were  addresed  in  Phase  I  and  are  also  included  in  the 
plans  for  Phase  II,  while  the  third  issue  will  be  addressed  later  in  the  project.  Laurie 
provided  a  summery  of  the  progress  to  date  which  included  the  identification  of  the  four 
performance  levels  (unacceptable,  marginal,  acceptable,  and  outstanding),  the  three  initial 
standard  setting  approaches  (soldier-based,  task-based,  and  critical  incident-based),  the 
development  of  the  initial  judgment  protocols,  and  the  Phase  I  workshops  that  tried  out 
these  protocol  on  three  MOS. 

Bob  Guion  expressed  concern  that  the  deHnition  of  "acceptable"  may  not  be  positive 
enough  and  wanted  to  know  what  the  Army’s  policy  was  concerning  encouraging  these 
soldiers  to  re-enlist.  Jane  said  that  although  the  Army  encourages  the  "acceptable" 
soldiers  to  re-enlist,it  is  more  concerned  with  targeting  extra  effort  and  limited 
resources  should  be  directed  at  encouraging  "outstanding"  soldiers  to  re-enlist. 


Setting  Individual  Standards 

Laurie  defined  two  major  issues  that  concerned  setting  individual  standards  and  two 
major  issues  that  addressed  combining  multiple  standards.  The  four  issues  were:  (1)  do 
different  methods  of  describing  performance  levels  affect  the  setting  of  performance 
standards?;  (2)  does  providing  practice  and/or  normative  information  affect  the  leniency 
of  performance  standards?;  (3)  how  do  we  combine  standards  set  for  individual 
performance  elements  into  standards  for  overall  performance;  and  (4)  are  there  judge 
characteristic  that  are  correlated  with  different  judgment  strategies/outcomes?  Laurie 
then  presented  result  that  addressed  the  first  two  issue  and  Phil  Szenas  presented  results 
that  addressed  the  final  two  issues. 
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Much  discussion  was  generated  concerning  the  task-based  standard  setting  protocol. 
Everyone  agreed  that  improvements  could  be  made  to  the  way  the  information  was 
presented.  Ron  Hambleton  suggesting  eliminating  the  "specific  steps  in  the  hands-on 
tests"  information  because  it  only  served  to  confuse  the  judges.  Joyce  and  Laurie 
suggested  provide  mock  hands-on  score  sheets  and  asking  SMEs  to  provide  standard 
setting  judgment  on  these  tasks.  John  suggested  simply  listing  the  tasks  covered  and 
showing  only  overall  percent-go  scores. 

Laurie  presented  three  conclusions  from  the  analyses  of  the  affects  of  using  different 
methods  of  describing  performance  levels  on  setting  of  performance  standards;  (1)  the 
different  description  methods  had  a  dramatic  effect  on  the  setting  of  standards;  (2)  the 
rating-based  method  resulted  in  somewhat  more  lenient  standards  for  minimal  perfor¬ 
mance;  and  (3)  the  task-based  method  resulted  in  very  strict  standards  for  minimal 
performance  on  many  dimensions. 

Laurie  then  presented  the  results  of  the  analyses  conducted  to  investigate  the  affects  of 
providing  practice  and  normative  information  may  have  had  on  the  leniency  of 
performance  standards.  The  conclusion  drawn  from  these  analyses  was  that  practice  and 
normative  information  did  not  have  an  effect  on  the  setting  of  performance  standards.  A 
caution  was  made  concerning  this  conclusion  due  to  the  small  sample  sizes  used  and  the 
limitation  of  the  manipulation. 


Combining  Multiple  Standards 

Phil  Szenas  presented  results  that  addressed  two  issues:  (1)  how  do  we  combine  multiple 
standards;  and  (2)  are  there  judge  characteristics  that  are  correlated  with  different 
judgment  strategies/outcomes.  He  described  the  conjoint  approach  used  in  the  workshops 
and  presented  results  on  the  modeling  of  judges’  decision  strategies. 

Four  conclusions  were  put  forward  relating  the  modeling  of  the  judges’  decision 
strategies:  (1)  judgments  of  overall  performance  ratings  are  well  represented  by  simple 
linear  unit  weighted  combination  of  equal  interval  performance  area  ratings;  (2)  rescaling 
the  performance  area  ratings  to  non-equal  interval  performance  area  ratings  improves  the 
modeling  of  raters’  judgments;  (3)  differential  weighting  of  the  performance  areas  leads 


to  further  improvement  in  the  modeling;  and  (4)  the  addition  of  a  term  to  capture  the 
spread  in  performance  ratings  further  improves  the  modeling  of  raters’  judgments. 

Next,  the  results  of  the  investigation  of  judge  characteristics  and  different  judgment 
strategies  were  presented.  The  conclusions  drawn  from  the  results  were:  (1)  that 
Officers  have  stricter  standards  than  NCOs;  (2)  that  judges  from  training  sites  have 
stricter  standards  than  judges  from  field  sites;  and  (3)  that  llBs  have  stricter  standards 
than  63Bs  and  71Ls. 

Phil  Bobko  suggested  going  back  and  conducting  the  rescaling  within  the  three  separate 
categories  in  order  to  determine  if  the  same  decision  models  would  apply  across  these 
groups. 

Linking  Enlistment  Standards  to  Job  Performance  Standards 

Jane  presented  a  proposed  approach  to  linking  job-specific  selection  standards  to  job- 
specific  performance.  She  pointed  out  that  other  screens  (e.g.  AFQT  medical,  possible 
ABLE  scores)  were  linked  to  more  general  components  of  job  performance.  She  showed 
how  empirical  data  could  be  used  to  build  expectancy  tables  and  how  the  expected 
performance  level  mix  would  vary  as  a  result  of  different  selection  standards. 

Phase  II  Data  Collection  Plans 

Laurie  and  John  summarized  the  plans  for  the  Phase  II  field  test  as  follows: 

Job  Component  Models 

*  the  main  focus  of  the  Phase  II  data  collection  is  to  see  if  the  encouraging 
results  found  for  the  three  MOS  studies  could  be  replicated  for  seven  addi¬ 
tional  MOS; 
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•  explore  the  possibility  of  constructing  a  Job  Component  model  based  on  a 
higher  level  of  generality  (fewer  components)  using  the  task-based  and 
activity-based  (or  both)  job  component  models; 

•  explore  alternative  methods  of  collection  relevance  ratings  beyond  importance; 

•  explore  difficulty,  especially  as  it  is  related  to  training  difficulty  and  standard 
setting;  and 

•  developing  a  questionnaire  to  measure  the  Judge’s  exposure  to  different  duty 
areas. 


Standard  Setting 

•  Modify  the  task-based  approach  to  included  a  simpler  presentations  and  clear 
presentation  of  step  information; 

•  explore  developing  a  critical  incident  based  instrument  that  can  generalize 
across  jobs;  and 

•  re-evaluate  the  presentation  of  normative  data. 

Phil  Bobko  reminded  the  group  that  it  is  not  too  early  to  be  thinking  about  selection  of 
MOS  and  judges  for  Phase  III. 
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Name: 


APPENDIX  B 

Phase  I  Task  Category  Taxonomy 


MOS  TASK  QUESTIONNAIRE 


There  are  96  tasks  in  this  questionnaire.  For  each  task,  we  would  like  you 
to  make  four  ratings.  First,  indicate  how  FREQUENTLY  each  task  is  performed  by 
soldiers  in  this  MOS,  using  the  following  FREQUENCY  rating  scale: 

0  =  Never;  this  task  is  not  part  of  the  job. 

1  =  Least  Often;  this  task  is  performed  much  less  often  than  most  other 

tasks. 

2  =  Not  Very  Often;  this  task  is  performed  less  often  than  most  other 

tasks. 

3  =  Often;  this  task  is  performed  about  as  often  as  other  tasks. 

4  =  Very  Often;  this  task  is  performed  more  often  than  most  other  tasks. 

5  =  Most  Often;  this  task  is  performed  much  more  often  than  most  other 

tasks. 


As  you  make  your  ratings,  think  about  soldiers  who  have  about  18 
months  of  service  in  this  MOS  after  Basic  and  AIT.  Also  keep  in  mind  all 
that  you  know  about  the  full  range  of  duty  assignments  for  this  MOS. 


After  you  have  made  FREQUENCY  ratings  for  all  96  tasks,  go  through  the  list 
again,  this  time  rating  the  IMPORTANCE  of  each  task  for  successful  performance  in 
three  different  areas  of  the  job:  Core  Technical  Area,  General  Soldiering  Area, 
and  Overall  Performance.  The  definitions  of  these  performance  areas  are  on  a 
separate  sheet,  entitled  PERFORMANCE  AREA  DEFINITIONS.  Please  read  these  defini¬ 
tions  carefully  before  making  your  IMPORTANCE  ratings. 

You  will  make  IMPORTANCE  ratings  using  the  following  rating  scale: 

0  =  No  Importance 

1  «  Extremely  Low  Importance 

2  «  Low  Importance 

3  =  Moderate  Importance 

4  =  High  Importance 

5  =  Extremely  High  Importance 

NOTE:  If  you  decided  that  a  particular  task  is  not  part  of  this  MOS  (so  you 

gave  it  a  FREQUENCY  rating  of  0),  you  should  leave  all  three  IMPORTANCE 

ratings  blank. 
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Please  look  at  the  EXAMPLES  below  and  read  through  their  explanations  before 
starting  to  make  your  ratings. 

1 


a.  Perform  medical  laboratory  procedures  -  conduct 
various  types  of  blood  tests,  unnalysis,  cultures,  etc. 

b.  Perform  operator  checks  and  services  on  weapons  - 
check,  disassemble,  assemble,  clean,  lubricate,  and 
adjust  weapons,  including  pistols,  rifles,  machineguns, 
hand  grenades,  and  breechblocks. 


Joe  supervises  94B  (Cooks).  He  went  through  the  list  of  tasks  and 
made  FREQUENCY  ratings  of  each  one,  then  returned,to  the  beginning  of  the 
list  and,  after  carefully  reading  the  PERFORMANCE  AREA  DEFINITIONS,  has 
started  making  h’is  ratings  of  how  IMPORTANT  each  task  is  for  successful 
performance  in  three  different  areas  of  the  job. 

1,  Since  he  felt  that  Task  a.,  "Perform  medical  laboratory  procedures," 
was  not  part  of  the  job  for  94B,  he  gave  this  task  a  FREQUENCY 
rating  of  0,  and  left  all  three  IMPORTANCE  ratings  blank. 

2.  Joe  felt  that  Task  b.,  "Perform  operator  checks  and  services 

on  weapons,"  was  performed  much  less  often  than  most  tasks  in  MOS 
94B,  so  he  gave  it  a  FREQUENCY  rating  of  1,  for  Least  Often. 

Joe  decided  that  this  task  was  of  Low  Importance  for  Core  Technical 
Area,  so  he  gave  it  an  IMPORTANCE  rating  of  2. 


For  General  Soldiering,  Joe  felt  that  this  task  was  of  High  Importance, 
so  he  gave  it  an  IMPORTANCE  rating  of  4  for  this  performance  area. 

For  overall  performance,  Joe  gave  this  task  a  rating  of  4  indicating 
the  task  was  of  High  Importance  for  overall  job  performance. 


Keep  the  PERFORMANCE  AREA  DEFINITIONS  handy  and  refer  to  them  as  often  as 
necessary  while  making  your  IMPORTANCE  ratings. 

NOTE:  Many  of  the  task  definitions  in  this  questionnaire  contain  specific 

examples  to  help  explain  and  clarify  the  task.  Please  keep  in  mind  that 
these  are  just  some  of  the  possible  examples;  it  was  not  practical  to  list 
every  possible  example. 


EXAMPLES: 


rnrgoEHCT 

IMPORT3LNCZ 

0 

•  N«v«r 

0 

No  laporranc* 

1 

•  L*asz  Often 

1 

■ 

Extremely  Lev  Importance 

2 

■  Not  Very  Often 

2 

• 

Low  laporcanc* 

3 

•  Often 

3 

Moderate  Importance 

4 

•  Very  Often 

4 

High  laporcanco 

5 

-  Most  Often 

5 

EXhraBoiy  High  laporhanca 
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TASK  QUESTIONNAIRE 


rRSOCTKCY 

0  «  Hever  0  ■ 

1  -  Least  Often  1  - 

2  •  Hot  Very  Often  2  - 

3  -  Often  3  - 

4  »  Very  Often  4  - 

5  -  Kcst  Often  5  - 


IMPORTAMCE 
No  lEDorCance 
Extrenely  Low  loportance 
Low  Icportance 
Moderate  Isportanca 
High  Icportance 
Extremely  High  Importance 


I.  MAD^rrENANCE 
A.  Mechanical  Systems  Maintenance 

1.  Perform  operator  maintenance  checks  and  services  — 
follow  directions  in  Operator’s  Manual;  conduct  before, 
during,  after,  and  weekly  operator  checks  and  services 
on  vehicles,  trailers,  generators,  construction  equip¬ 
ment,  or  other  kinds  of  mechanical  apparatus. 

2.  Perform  operator  checks  and  services  on  weapons  — 
check,  disassemble,  assemble,  clean,  lubricate,  and 
adjust  weapons,  including  pistols,  rifles,  machineguns, 
hand  grenades,  and  breechblocks. 

3.  Troubleshoot  mechanical  systems  -  measure,  use 
specialized  test  equipment  and  manuals,  and  observe 
mechanical  equipment  (for  example,  engines,  transmis¬ 
sions,  brakes,  hydraulics,  refrigeration  systems,  etc.)  to 
detect  and  diagnose  problems  and  malfunctions. 

4.  Repair  weapons  -  after  the  cause  of  a  problem  in  a 
weapon  has  been  found,  fix  it  using  the  appropriate 
tools  and  necessary  replacement  parts  by  lollowing 
directions  in  the  weapon’s  technical  manual. 

5.  Repair  mechanical  systems  -  after  the  cause  of  a 
problem  in  a  mechanical  part  has  been  found,  fix  it 
using  the  appropriate  tools  (for  example,  wrenches, 
screwdrivers,  gauges,  hammers,  soldering  equipment, 
etc.)  and  necessary  replacement  parts  by  following 
directions  in  the  equipment’s  technical  manual. 

6.  Troubleshoot  weapons  -  find  the  cause  of  malfunctions 
in  weapons  using  technical  manuals,  tools,  and  test 
equipment. 


B.  Electrical  and  Electronic  Systems  Maintenance 

7.  Install  electronic  components  -  connect  electronic  and 
communications  equipment  (for  example,  radios,  antea 
nas,  telephones,  teletypewriters,  radar,  power  supplies, 
etc.)  ana  check  system  for  operation. 
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rREQCENCY 

0  "  Never  0 

1  Least  0£ten  1 

2  "  Not  Very  0^ten  2 

3  “  Oiten  3 

4  “  Very  Oiten  4 

5  -  y.ost  Oiten  5 


IMPORT  ANCe 

•  No  Importance 

-  Extremely  Low  Importance 

-  Low  Importance 

•  Moderate  Importance 

-  High  Importance 

•  Extremely  High  Importance 


8.  Inspect  electrical  systems  ~  measure,  use  specialized 
test  equipment  and  manuals,  and  observe  electrical 
systems  (for  example,  generators,  wiring  harnesses, 
switches,  relays,  circuit  breakers,  etc.)  to  detect  and 
diagnose  pro6lems  and  malfunctions. 

9.  Inspect  electronic  systems  -  measure,  use  specialized 
test  equipment  and  manuals,  and  observe  electronic 
systems  (for  example,  communications  equipment,  radar, 
rhissile  and  tank  ballistics  computer,  etc.)  to  detect  and 
diagnose  problems  and  malfunctions. 

10.  Repair  electrical  systems  -  after  the  cause  of  an 
electrical  problem  has  been  found,  fix  it  with  the  ap¬ 
propriate  tools  (for  example,  wire  strippers,  pliers, 
soldering  irons,  etc.)  and  necessaiy  replacement  parts 
by  following  directions  in  the  equipment’s  technical 
manual. 

1 1 .  Repair  electronic  components  -  after  the  cause  of  an 
electronics  problem  has  been  found,  fix  it  with  ap¬ 
propriate  tools  (for  example,  test  sets,  screwdrivers, 
pliers,  soldering  guns,  etc.)  and  necessa^  replacement 
parts  by  following  directions  in  the  equipment’s  techni¬ 
cal  manual. 


n.  GENERAL  OPERATIONS 
C  Pack  and  Load 

12.  Pack  and  load  materials  -  load  and  lash  materials  onto 
transport  vehicles  (land,  sea,  or  air)  to  secure  and 
protect  from  damage  or  loss  during  shipment 

13.  Prepare  parachutes  -  inspect  cargo  and  personnel 
parachutes,  repair  or  replace  faulty  components,  and 
pack  parachutes  for  air  drop. 

14.  Prepare  equipment  and  supplies  for  air  drop  -  build  or 
assemble  platforms,  cushion's,  and  riggings  for  para¬ 
chuting  supplies,  equipment,  and  vehicles. 
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rKQnENCY 

0  “  Never  0 

1  »  Least  Olten  1 

2  -  Not  Very  Often  2 

3  -  Often  3 

4  “  Very  Often  4 

5  •  Most  Often  5 


IMPORTANCE 
•  No  laportancc 
>  Extremely  Low  Importance 

-  Low  Importance 

-  Moderate  Importance 
■  High  Importance 

-  Extremely  High  Importance 


D.  Vehicle  and  Equipment  Operations 

15.  Operate  power  excavating  equipment  -  use  air  ham¬ 
mers  and  drills,  paving  breakers,  grinders,  backfill 
tampers,  or  other  hand  operated  power  equipment  in 
building  concrete,  stone,  or  other  structures  (for 
example,  roads,  fortifications,  buildings,  etc.). 

16.  Operate  wheeled  vehicles  -  drive  wheeled  vehicles 
over  roads  and  cross-country  in  response  to  mission, 
terrain,  and  traffic  regulations. 

17.  Operate  track  vehicles  -  drive  track  vehicles  (for 
example,  tank,  APC,  BFV,  etc.)  in  response  to  mission, 
terrain,  and  traffic  controls. 

18.  Operate  boats  -  drive  boats  and  rafts. 

19.  Operate  lifting,  loading,  and  grading  equipment  - 
operate  fork  lifts,  cranes,  back-hoes,  graders,  and 
other  heavy  equipment  to  load,  unload,  or  move  heavy 
equipment,  supplies,  construction  materials  (for  ex¬ 
ample,  culvert  pipe,  building  and  bridge  parts),  or 
terrain  (for  example,  earth,  rocks,  trees,  etc.). 


E.  Construct/Assemble 

20.  Paint  -  prepare  surfaces  (clean,  remove  old  paint, 
sand)  and  apply  paint  with  brush,  roller,  or  spray. 

21.  Install  wire  and  cables  -  string  or  lay,  and  connect 
electrical  wire  or  communications  cables. 

22.  Repair  plastic  and  fiberglass  -  fix  plastic  or  fiberglass 
parts  and  structures  by  cutting,  sawing,  drilling,  sand¬ 
ing,  filling,  gluing,  and  painting. 

23.  Repair  metal  -  fix  metal  structures  or  parts  by  bend¬ 
ing,  cutting,  drilling,  welding,  hammering,  grinding, 
soldering,  and  painting. 

24.  Assemble  steel  structures  -  erect  bridges,  antennas, 
and  other  steel  structures.  May  require  the  assistance 
of  others  and  use  of  heavy  equipment. 


B-5 


rRTQCENCY 

0  “  Never  0 

1  -  Least  Often  1 

2  "  Not  Very  Often  2 

3  «  Often  3 

*  “  Very  Often  * 

5  ■  Most  Often  5 


IMPORTANCE 

-  No  Importance 

■  Extremely  Low  Importance 

-  Low  Importance 

-  Moderate  Importance 

-  High  Importance 

-  Extremely  High  Importance 


25.  Install  pipe  assemblies  -  place,  connect,  and  test  pipe 
assemblies  and  fixtures  (for  example,  plumbing,  POL 
pipelines  and  pumps,  etc.). 

26.  Construct  wooden  buildings  and  other  structures  — 
measure,  saw,  nail  or  plane  to  frame,  sheath,  and  roof 
buildings,  or  erect  trestles,  bridges,  and  piers. 

27.  Construct  masonry  bufldings  and  structures  -  measure, 
lay  brick  or  concrete  blocks,  or  build  forms  and  pour 
concrete  to  construct  walls,  columns,  field  fortifica¬ 
tions,  and  other  concrete  or  masonry  structures. 


F.  Technical  Procedures 

28.  Operate  gas  and  elecyic  powered  equipment  -  operate 
electric  generators,  air  compressors,  smoke  generators, 
quarry  machines,  mobile  washing  machines,  water 
pumps,  etc.,  to  produce  power  or  process  materials. 

29.  Select  lavouL  and  clean  medical  or  dental  equipment 
and  supplies  -  pi’epare  treatment  areas  for  use  by 
following  prescribed  procedures  for  laying-out  instru¬ 
ments  and  equipr.rent;  clean  equipment  and  area  for 
future  use. 

30.  Use  audiovisual  equipment  -  use  cameras  and 
videotape  to  record  sights  and  sounds  for  intelligence 
analysis,  training,  or  documentation. 

31.  Reproduce  printed  material  -  operate  duplicating 
machines,  offset  oresses,  and  similar  equipment  to 
reproduce  printed  materials;  collate  and  bind  materials 
usmg  various  typ  -s  of  bindery  equipment. 

32.  Operate  electronic  equipment  -  set  and  adj ust  the 
controls  to  operate  electronic  equipment  (for  example, 
radio,  computer  hardware,  missile  ballistics  controls, 
etc). 

33.  Operate  radar  -  operate  radar  equipment  and  interpret 
signals. 

34.  Operate  computer  hardware  -  operate  computer  hard¬ 
ware  such  as  tape  and  disk  drives,  optical  scanners, 
terminals,  and  other  input/output  devices.  (Does  not 
include  programming.) 
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FliSQCZUCY 

IMPORTXNCr 

0 

V 

Never 

0 

m 

No  Importance 

1 

m 

Least  orte.n 

1 

m 

Ejctremely  Low  Importance 

2 

m 

Net  Very  often 

2 

m 

Low  Importance 

3 

m 

Often 

3 

m 

Moderate  Importance 

4 

m 

Very  Often 

4 

m 

High  Importance 

5 

m 

Most  Often 

5 

m 

Extremely  High  Importance 

35.  Cook  “  prepare  food  and  beverages  according  to 
recipes  and  meal  plans  (measure,  mix,  bake,  etc); 
inspect  fresh  food  and  staples  for  freshness:  clean 
equipment  and  work  area. 

36.  Perform  medical  laboratory  procedures  -  conduct 
various  types  of  blood  tests,  urinalysis,  cultures,  etc. 

37.  Conduct  land  surveys  -  survey  terrain  to  determine 
elevations,  azimuths,  and  distances  of  terrain  features; 
record  information. 

38.  Provide  medical  or  dental  treatment  -  give  medical 
attention  to  soldiers  in  the  field  or  in  medical  or 
dental  clinics,  or  give  veterinary  treatment  to  animals 
(for  example,  administer  injections,  take  blood  pres¬ 
sure,  change  sterile  dressings,  etc.);  does  not  include 
first  aid. 


G.  Make  Technical  Drawings 

39.  Sketch  maps,  overlays,  or  rano^  mrHs  -  use  standard 
symbols  to  make  sketches  of  terrain,  including  loca¬ 
tions  of  buildings  and  other  objects,  targets,  avenues 
of  approach,  and  maneuver  areas. 

40.  Produce  technical  drawings  -  use  drafting  and  drawing 
equipment  to  make  technical  drawings  and  blueprints. 

41.  Draw  maps  and  overlays  -  use  drafting,  graphics,  and 
related  techniques  to  draw  and  revise  maps  from  aerial 
photographs. 

42.  Draw  illustrations  -  use  pen,  pencil,  paint,  or  other 
media  to  make  free  hand  technical  drawings  and  U- 
lustrations.  (Does  not  include  range  cards,  sketched 
maps  or  other  field  expedient  drawings.) 


m.  ADMINISTRATIVE 


H-  Oerical 

43.  Type  -  type  information  using  a  typewriter,  teletype¬ 
writer,  keypunch,  or  computer  terminal. 
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FREQCrxcy 

IMPORTANCE 

0 

Never 

0 

a 

No  Importance 

1 

■ 

Least  c:ten 

1 

m 

Extremely  Low  Importance 

2 

Not  Ver-y  Often 

2 

m 

Low  Importance 

3 

■ 

Often 

3 

m 

Moderate  Importance 

4 

■ 

Very  often 

4 

m 

High  Importance 

5 

Most  Often 

s 

m 

Extremely  High  Importance 

44.  Prepare  technical  forms  and  documents  -  follow 
standardized  procedures  to  prepare  or  complete  forms 
and  documents  (for  example,  personnel  records,  legal 
briefs,  requisition  requests,  inspection  records,  etc). 

45.  Record,  file,  and  dispatch  information  -  collect,  copy, 
update,  sort,  inde>q  file,  and  retrieve  information  (for 
example,  mail,  training  rosters,  personnel  statistics, 
supply  inventories,  etc.). 

46.  Receive,  store,  and  issue  supplies,  equipment  and  other 
materials  -  inspect  materials  and  review  paperwork 
when  receiving  materials;  sor^  transport,  and  store 
materials;  issue  or  ship  materials  to  authorized  person¬ 
nel  or  units. 


1.  Communication 

47.  Use  hand  and  arm  signals  -  communicate  messages  and 
instructions  using  hand  and  arm  signals. 

48.  Read  technical  manuals,  field  manuals,  regulation.^,  and 
other  publications  -  use  index  and  table  of  contents 

to  find  location  of  needed  information;  locate  informa¬ 
tion;  read  instructions,  diagrams,  charts,  and  tables. 

49.  Use  maps  -  read  and  interpret  map  symbols  and 
identic'  terrain  features  in  order  to  onent  map  to  your 
position  in  the  field;  determine  grid  coordinates; 
determine  directions;  identify  roads,  towns,  etc 

50.  Send  and  receive  radio  messages  -  use  standardized 
radio  codes  and  procedures  to  transmit  and  receive 
messages  and  other  information. 

51.  Give  short  oral  reports  -  use  standard  communication 
procedures  to  organize  and  deliver  information  (for 
example,  SALUTE,  call  for  and  adjust  indirect  fire, 
status  reports,  etc). 

52.  Receive  clients,  patients,  guests  -  schedule,  greet,  and 
give  routine  information  to  persons  seeking  medical, 
dental,  legal,  or  counseling  services. 

53.  Give  directions  and  instructions  -  give  verbal  informa¬ 
tion,  instructions,  or  directions  to  others. 
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rRTQCENCY 

ZKPORTXNCZ 

0 

Never 

0 

■i 

No  laportance 

1 

m 

Least  Often 

1 

m 

Extremely  Low  Importance 

2 

m 

Not  Very  Often 

2 

m 

Low  Importance 

3 

m 

Often 

3 

m 

Moderate  Importance 

4 

m 

Very  Often 

4 

m 

High  Importance 

5 

■ 

Host  Often 

5 

m 

Extremely  High  Importance 

54.  Write  and  deliver  presentations  -  make  formal  presen¬ 
tations,  for  example,  briefings,  radio  and  television 
broadcasts,  etc. 

55.  Interview  -  gather  information  from  clients,  patients, 
witnesses,  prisoners,  or  other  persons. 

56.  Provide  cnnnselino  and  other  interpersonal  interven¬ 
tions  -  conduct  personal  adjustment  counseling  with 
individuals  and  groups;  use  interpersonal  relations  skills 
to  solve  relationship  problems. 

57.  Write  documents  and  correspondence  -  draft  letters, 
reports,  memos,  etc.;  proofread  and  edit 


J.  Analyze  Information 

58.  Decode  data  -  use  coding  systems  and  rules  to 
decipher  and  interpret  coded  information  (for  example, 
use  CEOI,  interpret  symbols/signs,  etc.). 

59.  Analyze  electronic  signals  -  analyze  electronics  signals 
to  detect  threat  transmitters  and  electronic  counter¬ 
measures. 

60.  Analyze  weather  conditions  -  determine  weather 
conditions  and  analyze  their  effects  on  tactical 
operators. 

61.  Order  equipment  and  supplies  -  determine  needs  and 
requisition  needed  supplies,  materials,  and  equipment 

62.  Estimate  time  and  cost  of  maintenance  operations  - 
estimate  equipment  downtime  and  cost  of  repairs, 
including  parts  and  labor. 

63.  Plan  placement  or  use  of  tactical  equipment  -  using 
maps  and  on-site  inspection,  identify  positions  and 
areas  to  be  used  for  cover  and  concealment  and  to 
place  weapons,  fortifications,  mines,  and  detectors. 

64.  Translate  foreign  languages  -  translate  written  or 
spoken  foreign  language  communications. 
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FRTQOEKCX 

IMPORTANCE 

0 

“  Never 

0 

«  No  laportance 

1 

~  Least  Often 

1 

“  Extremely  Low  Importance 

2 

“  Not  Very  Often 

2 

-  Low  laportance 

3 

“  Often 

3 

•  Moderate  Importance 

4 

“  Very  Often 

4 

•  High  Importance 

5 

«  Most  Often 

5 

“  Extremely  High  Importance 

65.  Analyze  ipteUigence  data  —  determine  importance  and 
reliability  of  information;  use  information  to  determine 
identity,  capabilities,  disposition,  and  movement  of 
enemy  forces. 


K.  Applied  Math  and  Data  Processing 

66.  Control  money  -  keep  accounting  records;  disperse  and 
collect  money  and  money  orders. 

67.  Determine  firing  data  for  indirect  fire  weapons  —  use 
maps,  firing  charts,  and  targeting  and  ballistics  infor¬ 
mation  to  determine  elevation  and  azimuth  needed  for 
engaging  targets. 

68.  Compute  statistics  or  other  mathematical  calculations 
-  select  formulas  and  make  mathematical  calculations, 
with  or  without  using  calculators  or  computers;  report 
results. 

69.  Provide  programming  and  data  processing  support  for 
computer  operations  -  analyze  data  processing  needs; 
select  or  prepare,  edit,  test,  and  run  computer 
programs;  document  process  and  residts. 


I-  Control  Air  TraflQc 

70.  Control  air  traffic  -  coordinate  departing,  en  route, 
arriving,  and  holding  aircraft  by  monitoring  radar 
equipment,  communicating  with  aircraft  and  other  air 
traffic  control  units. 


IV.  COMBAT 

M.  Individual  Combat 

71.  Use  hand  grenades  -  identify,  inspect,  arm,  throw, 
and  secure  hand  grenades. 

72.  Protect  against  NBC  ha7j>rri<  ~  use  protective  clothing, 
masks,  and  decontamination  equipment  to  protect  self, 
others,  eauipment,  and  supplies  from  nuclear, 
biological,  and  chemical  hazards. 


Importance 


rRSQUENCY 

0  “  Never  0 

1  •  Least  Often  1 

2  «  Not  Very  Often  2 

3  ■  Often  3 

4  «  Very  Often  4 

5  -  Most  Often  5 


IMPORTANCE 
•  No  Importance 
"  Extremely  Low  Importance 

-  Low  Importance 

-  Moderate  Importance 

-  High  Importance 

-  Extremely  High  Importance 
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73.  Handle  demolitions  or  mines  -  store,  place,  charge, 
discharge,  and  disarm  explosives,  demolition  devices,  or 
mines. 

74.  Engage  in  hand-to-hand  combat  -  use  offensive  and 
defensive  maneuvers  to  overcome  hostile  individuals. 

75.  Fire  individual  weapons  -  aim,  track  and  fire  in¬ 
dividual  weapons,  such  as  rifles,  pistols,  machineguns, 
and  LAW,  at  designated  targets;  load,  reduce  a  stop¬ 
page,  and  clear  weapons. 

76.  Control  individuals  and  crowds  -  perform  guard  duty, 
including  challenge  and  password;  apprehend  and  search 
suspected  criminals  or  enemy  soldiers;  guard  prisoners; 
participate  in  riot  control. 

77.  Customs  and  laws  of  war  -  use  knowledge  of  Geneva 
convention  and  military  SOP  concerning  treatment  of 
enemy  personnel,  engagement  of  the  enemy,  conduct  of 
milit^  protocol  and  ceremony,  guard  duty,  and 
physical  readiness. 

78.  Navigate  -  during  the  day  or  night,  with  or  without  a 
map,  locate  positions  and  move" from  point  to  point  in 
response  to  terrain  features  (for  example,  for  cover  or 
concealment),  battle  conditions,  and  mission. 

79.  Survive  in  the  field  -  select,  prepare  and  occupy 
individual  tactical  positions  (for  example,  battle  posi¬ 
tions,  overwatch  positions,  observation  posts), 
camouflage  self  and  equipment,  observe  security 
procedures. 

80.  Move  and  react  in  the  field  -  move  on  foot  in  the 
battlefield  as  a  member  of  a  tactical  operation;  react 
to  threats,  including  direct  and  indirect  fire. 


N.  Crew-served  Weapons 

81.  Load  and  unload  field  artillery  or  tank  guns  -  operate 
breech  controls  and  handle  ammunition  (stow  and  load) 
to  prepare  guns  for  firing;  unload  or  extract  unused 
rounds  or  misfires. 
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0  •’  Never  0 

1  -  Least  Often  1 

2  “  Not  Very  Often  2 

3  -  Often  3 

4  “  Very  Often  4 

5  «  Host  Often  5 
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“  Extrenely  l/ow  Importance 
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82.  Fire  heavy  direct  fire  weapons  (for  example,  tank  main 
guns,  TOW  missile,  IFV  cannon,  etc)  --  using  weapon 
sights,  manipulate  weapon  controls  to  aim,  track,  and 
fire  on  targets. 

83.  Prepare  heavy  weapons  for  tactical  use  -  position  and 
prepare  for  firing  heavy  tactical  weapons,  such  as 
missiles,  field  artillery,  and  anti-aircraft  systems. 

84.  Place  and  camnnflage  tactical  equipment  and  materials 
in  the  field  -  place  mines,  detectors,  chemicals,  and 
camouflage  materials  into  position  in  the  battlefield. 

85.  Fire  indirect  fire  weapons  (for  example,  field  artilleiy 
and  heavy  mortars)  -  lay  weapon  by  adjusting  azimuth 
and  elevation  controls  in  response  to  fire  commands. 


O.  Give  First  Aid 

86.  Give  first  aid  ~  carry  out  first  aid  procedures  (for 
example,  CPR,  put  on  field  dressing,  prevent  shock, 
etc.). 


P.  Identify  Targets 

87.  Detect  and  identify  targets  -  with  or  without  optical 
devices  (for  example,  night  sights,  weapon  sights, 
binoculars,  etc),  locate  possible  targets,  and  identify 
type  (for  example,  troops,  tanks,  aircraft,  etc)  and 
nomenclature. 


Q.  Supervision 

88.  Plan  operations  -  plan,  prepare,  and  develop  orders 
for  team  operations,  includmg  combat,  support,  and 
technical  operations. 

89.  Direct/lead  teams  -  direct  combat  and  security  team 
activities  in  the  field  (for  example,  lead  reconnaissance 
teams,  set  up  offensive  and  defensive  positions,  carry 
out  a  fire  mission,  etc.). 
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FREQCSJCY 

0  ^  Never  0 

1  •  Least  c:ten  1 

2  “  Not  Very  Often  2 

3  -  Often  3 

*  ■  Very  often  4 

5  “  Most  Often  5 


IMPORTANCE 

•  No  Zoportanee 

-  Extremely  Low  Importance 

-  Low  Importance 

-  Moderate  Importance 

•  High  Importance 

-  Extremely  High  Importance 


90.  Monitor/inspect  -  monitor  subordinates  to  ensure 
that  they  are  carrying  out  their  duties  properly,  that 
they  have  the  correct  equipment,  that  supply  levels  are 
ade’quate  for  the  mission,  and  that  records  are  com¬ 
plete,  etc.). 

91.  Lead  -  influence  subordinates  by  setting  goals,  main¬ 
taining  good  lines  of  communication,  sharing  hardships, 
building  trusts,  etc. 

92.  Act  as  a  model  -  show  subordinates  correct  way  to 
perform  technical  tasks,  maintain  a  positive  attitude 
under  adverse  conditions,  demonstrate  proper 
military  bearing,  etc.,  on  a  day-to-day  basis. 

93.  Counsel  -  provide  individual  subordinates  with  support, 
assistance,  and  feedback  on  specific  performance, 
personal,  or  disciplinary  problems. 

94.  Cjimmunicate  -  compose  orders,  brief  subordinates  on 
things  that  are  happening  in  the  unit;  keep  superiors 
and  peers  informed,  etc. 

95.  Train  -  schedule,  plan,  and  conduct  training  for 
subordinates. 

96.  Personnel  Administration  -  prepare  and  conduct  per¬ 
formance  appraisals,  recommend  various  personnel 
actions,  keep  and  maintain  personnel  and  administrative 
records,  eta 
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Initial  Job  Activity  Taxonomy 

A.  Interpersonal  Behaviors. 

1.  Participation  in  work  teams. 

a.  Behaviors  facilitating  group  maintenance  and  morale  (e.g., 
being  humorous,  showing  support  for  other  team  members, 
offering  encouragement,  etc.). 

b.  Behaviors  facilitating  accomplishment  of  group  goals  (e.g., 
keeping  group  on  path  toward  goal,  frequently  restating 
goal,  making  substantive  suggestions,  etc.). 

c.  Behaviors  facilitating  conflict  resolution. 

2.  Peer  leadership  and  support  (one-on-one). 

a.  Coaching  peers. 

b.  Giving  direction/goal  setting. 

c.  Giving  reinforcement,  support,  encouragement. 

3.  Leadership  and  supervision. 

a.  Instructing/coaching,  (one-on-one) 

b.  Goal  setting/planning,  (one-on-one) 

c.  Administering  rewards/punishments,  (one-on-one) 

d.  Counseling  on  personal  problems,  (one-on-one) 

e.  Leading/directing/managing  the  team. 

f.  Resolving  conflicts  among  subordinates. 

B.  Speaking  behaviors. 

1-  Making  oral  reports. 

a.  Informal/one-on-one. 

b.  To  groups. 

2.  Selling  or  persuading  via  oral  communication. 

a.  One-on-one. 

b.  To  groups. 

3.  Instructing  via  oral  communication. 

a.  One-on-one. 

b.  To  groups. 

4.  Spontaneous  contribution  to  group  discussions/problem  solving. 

C.  Writing. 

1.  Writing  reports,  memos,  etc. 

a.  Short  messages  -  frequently. 
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b.  Longer  formal  reports  -  infrequently. 

2.  Writing. persuasive/sales  messages. 

a.  On  an  informal  daily  basis. 

b.  Formal  documents  for  clients,  vendors,  other  parts  of  the 
organization,  etc. 

3.  Creating  journalistic  or  literary  written  products  (e.g..  Army 
Times) . 

D.  Information  Processing 

1.  Process  verbal  information. 

a.  Monitor/recognize/identify  verbal  message. 

-single  source 

-multiple  source 

b.  Edit,  proof,  check  verbal  message 

c.  Recall  verbal  information 

d.  Interpret/evaluate  verbal  information 
-single  source 

-multiple  source 

2.  Process  numerical/quantitative  information. 

a.  Monitor/recognize/identify  quantitative  information. 

-single  source 

-multiple  source 

b.  Edit,  proof,  check  quantitative  message. 

c.  Recall  quantitative  information. 

d.  Interpret/evaluate  quantitative  information 
-single  source 

-multiple  source 

3.  Processing  figural  information  (Images,  pictures,  graphs,  etc.) 

a.  Monitor/recognize/ identify  figural  information 

-single  source 

-multiple  source 

b.  Edit,  proof,  check  figural  information 

c.  Recall  figural  information 

d.  Interpret/evaluate  figural  information 

4.  Processing  static  spatial  relations  (configural)  information  (the 
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parts  are  not  moving  -  e.g..  If  truck  cab  moves,  where  will 
trailer  go?) . 

a.  Monitor/recognize/identify  spatial  information 

b.  Recall  spatial  information 

c.  Interpret/evaluate 

5.  Processing  dynamic  spatial  information  (the  parts  are  moving 
e.g.,  two  vehicles  are  converging). 

a.  Monitor/recognize/identify  dynamic  spatial  information. 

b.  Recall  dynamic  spatial  inf  mation. 

c.  Interpret/evaluate  dynamic  spatial  information 

E.  Complex  Problem  Solving/Troubleshooting  Behavior 

1.  Electrical  equipment  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

2.  Mechanical  equipment  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

3.  Tactical  maneuver  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

4.  Logistical  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

5.  Administrative  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

6.  Leadership/supervision  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 
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c.  Evaluate  and  choose  solution 

7.  Mental/physical  health  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

8.  Communication  problems 

a.  Identify  problem 

b.  Generate  possible  solutions 

c.  Evaluate  and  choose  solution 

F.  Operating  Equipment 

1.  Operating  hand  held  equipment 

a.  Precision  tools/equipment/weapons 

b.  Non-precision  tools/equipment/weapons 

2.  Operating  keyboard  device 

a.  Numeric  keyboard 

b.  Full  typewriter  or  computer  keyboard 

3.  Adjusting  control  devices 

a.  Single  limb 

b.  Multiple  limb 

4.  Operating  large  power  assisted  equipment 

a.  Single  limb 

b.  Multiple  limb 

5.  Driving  heavy  equipment 

a.  Wheeled  vehicles 

b.  Tracked  vehicles 

6.  Driving  light  vehicles  (car,  jeep,  quarter  ton). 

a.  Frequently  or  over  long  distances 

b.  Infrequently 

7.  Operating  aircraft 

a.  Helicopter 

b.  Fixed  wing 

G.  Physical  Activities 

1.  Walking  long  distances 

C-4 


a.  Carrying  weight 

b.  Without  weight 

2.  Running  short  to  middle  distances 

3.  Pushing,  pulling,  lifting  heavy  weights 

4.  Throwing  hand  held  objects 

5.  Using  hands  to  fold,  sort,  feed,  etc. 

6.  Aiming  at  stationary  objects 

a.  Without  electronic  aids 

b.  With  electronic  aids 

7.  Aiming  at  moving  objects 

a.  Without  electronic  aids 

b.  With  electronic  aids 

8.  Working  long  hours  without  rest 

9.  Working  under  adverse  or  dangerous  conditions. 


Name; 


APPENDIX  D 

Phase  I  Job  Activity  Taxonomy 


MOS  ACTIVITY  QUESTIONNAIRE 

There  are  53  activities  in  this  questionnaire.  For  each  activity,  we  would 
like  you  to  make  four  ratings.  First,  indicate  how  FREQUENTLY  each  activity  is 
performed  by  soldiers  in  this  MOS,  using  the  following  FREQUENCY  rating  scale: 

0  =  Never;  this  activity  is  not  part  of  the  job. 

1  =  Least  Often;  this  activity  is  performed  much  less  often  than  most  other 

activities. 

2  Not  Very  Often;  this  activity  is  performed  less  often  than  most  other 
activities. 

3  =  Often;  this  activity  is  performed  about  as  often  as  other  activities. 

4  =  Very  Often;  this  activity  is  performed  more  often  than  most  other 

activities. 

5  =  Most  Often;  this  activity  is  performed  much  more  often  than  most  other 

activities. 


As  you  make  your  ratings,  think  about  soldiers  who  have  about  18  months 
of  service  in  this  MOS  after  Basic  ='nd  AIT.  Also  keep  in  mind  all  that  you 
know  about  the  full  range  of  duty  assignments  for  this  MOS. 


After  you  have  made  FREQUENCY  ratings  for  all  53  activities,  go  through  the 
list  again,  this  time  rating  the  IMPORTANCE  of  each  activity  for  successful 
performance  in  three  different  areas  of  the  job:  Core  Technical  Area,  General 
Soldiering  Area,  and  Overall  Performance.  The  definitions  of  these  performance 
areas  are  on  a  separate  sheet,  entitled  PERFORMANCE  AREA  DEFINITIONS.  Please 
read  these  definitions  carefully  before  making  your  IMPORTANCE  ratings. 

You  will  make  IMPORTANCE  ratings  using  the  following  rating  scale: 

0  »  No  Importance 

1  »  Extremely  Low  Importance 

2  »  Low  Importance 

3  -  Moderate  Importance 

4  ■  High  Importance 

5  ■  Extremely  High  Importance 

NOTE:  If  you  decided  that  a  particular  activity  is  not  part  of  this  MOS  (so 

you  gave  it  a  FREQUENCY  rating  of  0),  you  should  leave  all  three 
IMPORTANCE  ratings  blank. 
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Please  look  at  the  EXAMPLES  below  and  read  through  their  explanations  before 
starting  to  make  your  ratings. 


EXAMPLES: 


Importance 


a.  Interview  -  ask  frequent  questions,  as  in  mterview- 
ing  and  investigating. 

b.  Record  information  -  record  information  using  paper 
and  pencil  or  keyboard. 


Joe  supervises  94B.  He  went  through  the  list  of  activities  and  made 
FREQUENCY  ratings  of  each  one,  then  returned  to  the  beginning  of  the 
list  and,  after  carefully  reading  the  PERFORMANCE  DEFINITIONS,  has 
started  making  his  ratings  of  how  IMPORTANT  each  activity  is  for  successful 
performance  in  three  different  areas  of  the  job. 

1.  Since  he  felt  that  Activity  a.,  "Interview,"  was  not  part  of  the  job 
for  94B,  he  gave  this  activity  a  FREQUENCY  rating  of  0,  and  left  all 
three  IMPORTANCE  ratings  blank. 

2.  Joe  felt  that  Activity  b.,  "Record  Information"  was  performed 

much  less  often  than  most  activities,  so  he  gave  it  a  FREQUENCY  rating 
of  1,  for  Least  Often. 

Joe  decided  that  this  activity  was  of  Low  Importance  for  Core  Techni¬ 
cal  Area,  so  he  gave  it  an  IMPORTANCE  rating  of  2. 


For  General  Soldiering,  Joe  felt  that  this  activity  was  of  High  Impor¬ 
tance,  so  he  gave  it  an  IMPORTANCE  rating  of  4  for  this  performance  area. 

For  overall  performance,  Joe  gave  this  activity  a  rating  of  2 
indicating  the  activity  was  of  Low  Importance  for  overall  job  performance. 


0 _ 

I  H  2 


ntsQurjicY 

0  •  N*v«r  0 

1  *  L«ast  Oftan  1 

2  •  Ko^  Vary  Oftan  2 

3  •  Oftan  3 

4  »  Vary  Of  tan  4 

5  •  .Most  Oftan  5 


XXPORTXMC8 

*  No  Zmporcanea 

*  Extraaaly  Lov  laportanca 
■  Lov  laportanca 

*  Hodarata  laportanea 
*■  Hign  iBportanca 

*  Extraaaly  KlgA  Zaporcanea 


Keep  the  PERFORMANCE  DEFINITIONS  handy  and  refer  to  them  as  often  as 
necessary  while  making  your  IMPORTANCE  ratings. 

NOTE:  Many  of  the  activity  definitions  in  this  questionnaire  contain  specific 

examples  to  help  explain  and  clarify  the  activity.  Please  keep  in  mind 
that  these  are  just  some  of  the  possible  examples;  it  was  not  practical 
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JOB  ACnVITY  QUESTIONNAIRE 


rREQUUKCY 

0  “  Never  0 

1  »  Least  0;ten  1 

2  -  Not  Very  often  2 

3  «  Often  3 

4  •  Very  Often  4 

5  “  Most  Often  5 


IHFORTANC2 

•  No  Inportance 

«  Extremely  Low  Importance 

•  Low  Importance 

•  Moderate  Importance 
»  High  Importance 

«  Extremely  High  Importance 


A.  Leadership/Teamwork 

1.  Work  in  a  team  -  participate  in  work  or  combat  teams 
by  helping  resolve  conflicts,  keeping  the  team  moving 
toward  its  goals,  and  maintaining  morale  of  other 
group  members. 

2.  Lead  a  team  -  lead  or  direct  a  team  or  unit  as  it 
tries  to  accomplish  a  prescribed  mission  or  goal. 

3.  Support/advLse  peers  ~  support  and/or  advise  peers 
when  they  have  difficulty. 

4.  Support/advise  subordinates  -  support  and/or  advise  • 
subordinates  when  they  have  difficulty. 

5.  Coach  peers  -  instruct/train  peers  on  specific  job 
tasks. 

6.  Coach  subordinates  -  instruct/train  subordinates  on 
specific  job  tasks. 


B.  Communication 

7.  Make  oral  reports  fto  individuals^  ~  make  oral  reports 
to  other  individuals. 

8.  Make  oral  reports  fto  groups'^  -  give  oral  briefings  to 
a  group. 

9.  Relay  oral  instructions  -  relay  oral  instructions  on  job 
tasks  or  job  procedures  to  other  individuals  or  to 
groups. 

10  Interview  -  ask  frequent  questions,  as  in  interviewing 
and  investigating. 

11.  Record  information  -  record  information  using  paper 
and  pencil  or  keyboard. 

12.  Write  brief  messages  --  write  brief  messages  (for  ex¬ 
ample,  memos,  day-to-day  instructions,  business  letters, 
etc.). 

13.  Write  longer  reports  -  compose  and  write  longer 
formal  reports  or  articles  (tnat  is,  longer  than  one  or 
tw'o  pages). 


rRZQOENClt 

0  ■  Never  0 

1  •  Least  Often  1 

2  ■  Not  Very  Often  2 

3  »  Often  3 

4  “  Very  Often  4 

5  •  Most  Often  5 


IMPORTANCE 
"  No  iBportance 

-  Extreaely  Low  laportance 

-  Low  laportance 

a  Moderate  laportance 
a  High  laportance 
a  Extremely  High  Importance 


C  Use  Information 

14.  Monitor/interpret  verbal  messages  -  monitor,  identify, 
and/or  interpret  verbal  messages  (oral  or  written, 
obtained  from  radio,  teletype,  computer  terminal, 
correspondences,  etc.) 

15.  Recall  verbal  information  -  recall  verbal  information. 
(That  is,  to  perform  the  required  job  tasks,  the  in¬ 
dividual  must  rely  on  memory  for  necessary  verbal 
information  that  was  communicated  more  than  a  few 
hours  ago.  However,  this  item  does  not  refer  to 
information  presented  during  formal  training.) 

16.  Monitor/interpret  numerical  information  -  monitor, 
identify,  and/or  interpret  numerical/quantitative  infor¬ 
mation  that  is  presented  via  written  reports,  radio, 
CRT,  or  other  electronic  equipment 

17.  Recall  numerical  information  -  recall  numerical/quan¬ 
titative  information.  (Again,  "recall"  refers  to  the  fact 
that  the  job  requires  the  use  of  memo^  to  have  the 
necessary  information  to  perform  the  job  tasks. 
However,  it  is  nal  memory  for  training  content  but 
memory  for  information  acquired  as  part  of  the  job.) 

18.  Monitor/lnterpret  figural  information  -  monitor,  iden¬ 
tify,  and/or  interpret  figural  information  (for  example, 
CRT  images,  pictures,  graphs,  schematics,  sketches, 
terrain  features,  etc.). 

19.  Recall  figural  information  -  recall  figural  infonnation 
(for  example,  CRT  images,  p'  res,  graphs,  terrain 
features,  etc.). 

20.  Follow  oral  directions  -  follow  oral  directions  (for 
example,  directions  for  how  to  complete  documents, 
move  from  point  A  to  point  B,  or  schedule  repairs  on  a 
variety  of  equipment,  etc.). 

21.  Follow  written  directions  -  follow  written 
directions/instructions  as  part  of  normal  job  duties  (for 
example,  follow  written  orders,  SOP  protocols,  techni¬ 
cal  manuals,  etc.). 
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rHEQnENCY 

IMFORTANC8 

0 

Never 

0 

■ 

No  Isportance 

1 

Least  Often 

1 

m 

Extremely  Low  Importance 

2 

■ 

Not  Very  Often 

2 

m 

Low  Importance 

3 

■ 

Often 

3 

m 

Moderate  Imoortance 

4 

a 

Very  Often 

4 

m 

hi^h  Importance 

5 

m 

Most  Often 

5 

m 

Extremely  High  Importance 

D.  Perceptual  Judgements 

22.  Judge  size  and  distance  -  for  example,  make  judg¬ 
ments  of  relative  size  and  distance,  which  of  two 
objects  is  closer/larger,  the  distance  to  a  target,  etc. 

23.  Judge  location  -  orient  oneself  relative  to  location 
and  direction  from  physical  objects  or  terrain  features 
and  maintain  the  proper  orientation  while  moving  from 
point  to  point- 

24.  Judge  paths  of  moving  objects  -  make  judgments 
about  the  relative  position  of  moving  objects  (for 
example.  Where  will  two  vehicles  converge?  As  the 
truck  cab  moves,  where  will  the  trailer  go?). 


E.  Problem  SolvingiTroubleshoodng 

25.  Solve  electric^  system  problems  -  troubleshoot 
electrical  equipment  problems  (that  is,  identify  the 
proble  1  and  suggest  solution). 

26.  Solve  mechanic^  system  problems  -  troubleshoot 
mechanical  equipment  problems  (that  is,  identify  the 
problem  and  suggest  solution). 

27.  Solve  logistical  problems  -  identify  and  solve  logistical 
problems. 

28.  Solve  tactical  maneuver  problems  -  identify  and  solve 
tactical  maneuver  problems  (for  example,  positioning  of 
teams  in  the  field,  countering  moves  of  the  enemy, 
etc.). 

29.  Solve  administrative  problems  ~  identify  and  solve 
administrative  problems  (for  example,  correct  record 
keeping  errors,  personnel  scheduling,  etc). 

30.  Solve  leadership  problems  -  identify  and  solve 
leadership/supervision  problems  (for  example,  involving 
discipline,  training  needs,  conflicts  between  people, 
motivation  of  team  members,  etc.). 

31.  Solve  medical  problems  -  identify  and  solve  physical 
health  problems  (that  is,  problems  that  lequire  special 
training  beyond  basic  first  aid). 
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Importance 


rRTQOEKCY 

0  »  Never  0 

1  •  I^ast  Often  1 

2  “  Not  Very  Often  2 

3  «  Often  3 

4  »  Very  Often  4 

5  •  Most  Often  5 


IMFORTANCZ 

-  No  Importance 

•  Extremely  Low  Importance 

•  Low  Importance 

••  Moderate  Importance  - 

-  High  Importance 

-  Extremely  High  Importance 


Oi 

c 


o  c 


'IT  ^ 

Qj  ^ 
Qi 


f 

-V 
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32.  Solve  communication  problems  -  identify  and  solve 
communication  problems  (that  is,  not  technical/equip¬ 
ment  problems,  but  problems  resulting  from  inaccurate 
or  non-existent  communication). 


F.  Operate  Equipment 

33.  Operate  precision  hand-held  equipment  -  operate 
hand-held  equipment  requiring  great  precision  (for 
example,  synnge,  calipers,  soldering  or  welding  equip¬ 
ment,  etc.). 

34.  Operate  hand-held  tools  -  operate  hand-held  equip¬ 
ment  that  does  not  require  great  precision  (for  ex¬ 
ample,  hammer,  wrench,  etc). 

35.  Operate  band-held  power  equipment  -  operate  hand¬ 
held  power  assisted  equipment  (for  example,  electric 
saw,  electric  wrench,  etc). 

36.  Operate  large  power  equipment  -  operate  large  power 
assisted  equipment  (for  example,  forklift,  bulldozer, 
backhoe,  etc^ 

37.  Operate  full  keyboard  -  operate  full  typewriter  or 
computer  keyboard. 

38.  Operate  numeric  keyboard  -  operate  a  numeric 
keyboard  only  (that  is,  just  10  basic  keys,  0-9). 


G.  Adjust  and  Gjntrol 

39.  Adjust  device  using  one  limb  -  make  adjustments  in 
control  devices  by  using  just  one  hand  or  one  foot  (for 
example,  a  dial,  a  lever,  a  foot  pedal,  etc). 

40.  Adjust  control  device  using  multiple  limbs  -  make 
adjustments  in  control  devices  by  using  more  than  one 
limb  (for  example,  twist  a  cylinder  with  one  hand  while 
setting  a  dial  with  the  other,  depress  a  footpedal  while 
moving  equipment  into  proper  alignment  with  hands, 
etc). 
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rREQCEKCY 

IMPORTANCE 

0 

• 

never 

0 

No  laportance 

1 

Least  Olten 

1 

m 

Extreaely  X.ow  laportance 

2 

m 

Not  Very  Often 

2 

m 

Low  laportance 

3 

m 

Often 

3 

m 

Moderate  laportance 

4 

m 

Very  Often 

4 

« 

High  laportance 

5 

m 

Most  Often 

5 

■ 

Extreaely  High  laportance 

H.  Drive 

41.  Drive  tracked  vehicle  -  drive  heavy  vehicles  (tracked). 

42.  Drive  heavy  wheeled  vehicle  —  drive  heavy  vehicles 
(wheeled). 

43.  Drive  light  wheeled  vehicle  —  drive  light  wheeled 
vehicles. 


I.  Aiming 

44.  Aim:  stationary  target  —  aim  at  a  stationary  object 

45.  Aim:  moving  targ(^.^  —  manually  track  a  moving  target 


J.  Physical  Actions 

46.  Walk  long  distance!;  -  walk  long  distances  carrying 
significant  weight 

47.  Run  short  distance.^  -  run  short  to  middle  distances. 

48.  Push,  pull,  lift  heavyweights  -  push,  pull,  or  lift 
heavy  weights. 

49.  Throw  objects  ~  throw  hand-held  objects. 

50.  Sort  fold,  feed  bv  hand  ~  use  hands  to  fold  objects, 
sort  objects,  or  feed  objects  mto  a  machine. 

5 1.  Make  coordinated  movements  -  use  well  coordinated 
hand,  arm,  and  upper  body  movements  (for' example,  as 
in  setting  up  equipment  moving  hazardous  material 
quickly,  etc.). 

52.  Work  long  hours  -  work  long  hours  without  rest 

53.  Work  under  adverse  conditions  -  work  under  adverse 
or  dangerous  conditions. 
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APPENDIX  E 

Detailed  Results  for  Task  Category  and  Job  Activity  Analyses 


Phase  I  Field  Test 

Summary  of  Single-rater  Reliability  Estimates 

for  Task  Category  and  Job  Activity  Relevance  Ratings 

Across  3  MOS 


Task  Category  Job  Activity 

CTP  GSP  OJP  Pilot  CTP  GSP  OJP  Pilot 


All  Ratings 


Overall 

0.55 

0.46- 

0.52 

0.58 

0.43 

0.30 

0.37 

0.45 

Within  Rank 

0.60 

0.48 

0.54 

0.62 

0.51 

0.32 

0.41 

0.48 

Within  Command 

0.72 

0.52 

0.59 

0.63 

0.50 

0.32 

0.41 

0.47 

Within  Rank  &  Command 

0.74 

0.53 

0.60 

0.65 

0.54 

0.32 

0.43 

0.50 

NCO  Ratings 

Overall 

0.58 

0.50 

0.53 

0.61 

0.37 

0.30 

0.35 

0.49 

Within  Command 

0.59 

0.52 

0.54 

0.63 

0.37 

0.30 

0.35 

0.49 

Officer  Ratings 

Overal 1 

0.72 

0.50 

0.59 

0.63 

0.62 

0.33 

0.47 

0.58 

Within  Command 

0.85 

0.52 

0.61 

0.64 

0.63 

0.33 

0.48 

0.59 

FORSCOM  Ratings 

Overall 

0.72 

0.55 

0.60 

0.65 

0.39 

0.31 

0.37 

0.52 

Within  Rank 

0.75 

0.55 

0.61 

0.66 

0.43 

0.32 

0.39 

0.53 

DOTD  Ratings 

Overal 1 

0.51 

0.47 

0.51 

0.63 

0.46 

0.29 

0.38 

0.47 

Within  Rank 

0.54 

0.48 

0.53 

0.64 

0.50 

0.29 

0.39 

0.48 

Note  CTP  -  Core  Technical  Proficiency 

GSP  -  General  Soldiering  Proficiency 
OJP  -  Overall  Job  Proficiency 
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Variance  Components  Tables  Combining  all  3  MOS 

Task  and  Activity  Ratings  for: 

Core  Technical  Proficiency  (CTP) 

General  Soldiering  Proficiency  (GSP) 

Overall  Job  Performance  (OJP) 

CTP  for  NCOS 
CTP  for  Officers 
CTP  for  FORSCOM 
CTP  for  DOTD 

GSP  for  NCOS 
GSP  for  Officers 
GSP  for  FORSCOM 
GSP  for  DOTD 

OJP  for  NCOS 
OJP  for  Officers 
OJP  for  FORSCOM 
OJP  for  DOTD 


Phase  I  Test 

Variance  Components  for  Importance  Ratings 
for  Core  Technical  Proficiency 
(All  3  MOS) 


Task  Activity 


Components  Variance 

Percent  Variance 

Percent 

Component 

0.562 

18.40% 

0.113 

3.50% 

MOS 

0.230 

7.53% 

0.238 

7.38% 

Component  x  MOS 

0.897 

29.36% 

1.040 

32.24% 

Rank 

0.029 

0.95% 

0.082 

2.54% 

Command 

0.018 

0.59% 

0.076 

2.36% 

Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Command  x  MOS 

0.045 

1.47% 

0.032 

1.00% 

Rank  x  Command  x  MOS 

0.155 

5.07% 

0.328 

10.17% 

Component  x  Rank 

0,047 

1.54% 

0.096 

2.98% 

Component  x  Command 

0.005 

0.16% 

0.000 

0.00% 

Comp  X  Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.000 

0.00% 

0.011 

0.34% 

Comp  X  Command  x  MOS 

0.474 

15.52% 

0.032 

0.99% 

Cp  X  Rk  X  Cd  X  M 

0,000 

0.00% 

0.000 

0.00% 

Rater{Rk  Cd  M) 

0.030 

0.98% 

0.000 

0.00% 

Component  x  Rater 

0.563 

18.43% 

1.178 

36.51% 

Total 

3.055 

100.00% 

3.226 

100.00% 

RELIABILITY 


Overall 
Within  Rank 
Within  Command 
Within  Rank  &  Command 


0.55 

0.43 

0.60 

0.51 

0,72 

0.50 

0.74 

0.54 

I 
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Phase  I  Test 

Variance  Components  for  Importance  Ratings 
for  General  Soldiering  Proficiency 
(All  3  MOS) 

Task  Activity 


Components  Variance  Percent  Variance  Percent 


Component 

1.380 

41.21% 

0.700 

22.75% 

MOS 

0.048 

1.43% 

0.046 

1.49% 

Component  x  MOS 

0.115 

3.43% 

0.163 

5.30% 

Rank 

0.014 

0.42% 

0.015 

0.49% 

Command 

0.013 

0.39% 

0.000 

0.00% 

Rank  x  Command 

0.000 

0.00% 

0.037 

1.20% 

Rank  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Command  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  Command  x  MOS 

0.044 

1.31% 

0.086 

2.79% 

Component  x  Rank 

0.000 

0.00% 

0.037 

1.20% 

Component  x  Command 

0.062 

1.85% 

0.000 

0.00% 

Comp  X  Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.064 

1.91% 

0.008 

0.26% 

Comp  X  Command  x  MOS 

0.236 

7.05% 

0.079 

2.57% 

Cp  X  Rk  X  Cd  X  M 

0.000 

0.00% 

0.018 

0.58% 

Rater(Rk  Cd  M) 

0.267 

7.97% 

0.346 

11.24% 

Component  x  Rater 

1.106 

33.02% 

1.542 

50.11% 

Total 

3.349 

100.00% 

3.077 

100.00% 

RELIABILITY 


Overall 

0.46 

0.30 

Within  Rank 

0.48 

0.32 

Within  Command 

0.52 

0.32 

Within  Rank  &  Command 

0.53 

0.32 
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Phase  I  Test 

Variance  Components  for  Importance  Ratings 
for  Overall  Job  Performance 
(All  3  MOS) 


Task 

Activity 

Components 

Variance 

Percent  Variance 

Percent 

Component 

1.190 

35.63% 

0.430 

14.48% 

MOS 

0.080 

2.41% 

0.087 

2.93% 

Component  x  MOS 

0.468 

14.00% 

0.579 

19.51% 

Rank 

0.021 

0.63% 

0.046 

1 . 56% 

Command 

0.019 

0.57% 

0.022 

0.75% 

Rank  x  Command 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Command  x  MOS 

0.000 

0.00% 

0.000 

0.00% 

Rank  x  Command  x  MOS 

0.091 

2.73% 

0.173 

5.82% 

Component  x  Rank 

0.010 

0.29% 

0.067 

2.25% 

Component  x  Command 

0.000 

0.00% 

0.022 

0.74% 

Comp  X  Rank  x  Command  0.000 

0.00% 

0.000 

0.00% 

Comp  X  Rank  x  MOS 

0.019 

0.57% 

0.000 

0.00% 

Comp  X  Command  x  MOS 

0.292 

8.75% 

0.075 

2.53% 

Cp  X  Rk  X  Cd  X  M 

0.000 

0.00% 

0.004 

0.13% 

Rater(Rk  Cd  M) 

0.151 

4.52% 

0.137 

4.62% 

Component  x  Rater 

0.999 

29.92% 

1.327 

44.67% 

Total 

3.341 

sssssssa: 

100.00% 

Bssssaaaaa 

2.970 

asasBsxs 

100.00% 

RELIABILITY 


Overall 

0.52 

0.37 

Within  Rank 

0.54 

0.41 

Within  Command 

0.59 

0.41 

Within  Rank  &  Command 

0.60 

0.43 
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Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  Core  Technical  Proficiency 
(NCO  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

0.71 

24.97% 

0.29 

9.29% 

MOS 

0.16 

5.47% 

0.06 

1.94% 

Component*MOS 

0.79 

27.47% 

0.81 

25.83% 

Command 

0.05 

1.82% 

0.00 

0.00% 

Component*Command 

0.02 

0.53% 

0.00 

0.00% 

MOS*Command 

0.00 

0.00% 

0.01 

0.37% 

Rater(Cmd  MOS) 

0.31 

10.86% 

0.55 

17.51% 

Component*Rater(Cmd  MOS) 

0.82 

28.86% 

1.42 

45.06% 

2.86 

sssxsass: 

100.00% 

BSSSaSSKS 

3.15 

100.00% 

Rel iabil ity 

Rel iabil ity 

Overall 

0.58 

0.37 

Within  Command 

0.59 

0.37 

Phase  I  Test 


Variance  Components  for  Importance  Ratings  for  Core  Technical  Proficiency 
(Officer  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 

Variance  Percent  Variance  Percent 


Component 

0.45 

19.85% 

MOS 

0.16 

7.10% 

Component*MOS 

1.03 

45.02% 

Command 

0.00 

0.00% 

Component*Command 

0.00 

0.00% 

MOS*Command 

0.35 

15.15% 

Rater(Cd  M) 

0.00 

0.00% 

Component*Rater(Cd  M) 

0.29 

12.87% 

0.08 

2.84% 

0.29 

9.92% 

1.44 

49.18% 

0.01 

0.28% 

0.00 

0.00% 

0.05 

1.63% 

0.07 

2.55% 

0.98 

33.60% 

2.29  100.00% 


2.93  100.00% 


Reliability 


Reliability 


Overall  0.72 

Within  Command  0.85 


0.62 

0.63 
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Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  Core  Technical  Proficiency 
(FORSCOM  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

0.58 

18.49% 

0.12 

3.98% 

MOS 

0.23 

7.36% 

0.00 

0.00% 

Component*MOS 

1.45 

46.27% 

1.08 

35.29% 

Rank 

0.00 

0.00% 

0.00 

0.00% 

Component*Rank 

0.02 

0.59% 

0.06 

1.88% 

M0S*Rank 

0.08 

2.69% 

0.22 

7.23% 

Rater(Rk  M) 

0.23 

7.49% 

0.38 

12.50% 

Component*Rater(Rk  M) 

0.53 

17.09% 

1.19 

39.13% 

3.13 

100.00% 

3.05 

100.00% 

Reliability 

Reliability 

Overall 

0.72 

0.39 

Within  Rank 

0.75 

0.43 

Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  Core  Technical  Proficiency 
(DOTD  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

0.48 

19.14% 

0.04 

1.62% 

MOS 

0.09 

3.57% 

0.18 

6.46% 

Component*MOS 

0.72 

28.61% 

1.03 

37.85% 

Rank 

0.01 

0.20% 

0.00 

0.00% 

Component*Rank 

0.10 

3.94% 

0.16 

5.76% 

M0S*Rank 

0.04 

1.64% 

0.06 

2.10% 

Rater{Rk  M) 

0.19 

7.75% 

0.17 

6.10% 

Component*Rater(Rk  M) 

0.88 

35.15% 

1.09 

40.11% 

2.51 

100.00% 

2.73 

100.00% 

Overall 

Within  Rank 

Reliability 

0.51 

0.54 

Reliability 

0.46 

0.50 
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Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  General  Soldiering 
(NCO  ratings  only  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.29 

41.01% 

0.69 

22.07% 

MOS 

0.04 

1.25% 

0.00 

0.00% 

Component*MOS 

0.26 

8.19% 

0.25 

7.96% 

Command 

0.05 

1.54% 

0.00 

0.00% 

Component*Command 

0.04 

1.32% 

0.00 

0.00% 

MOS*Command 

0.00 

0.00% 

0.02 

0.59% 

Rater (Cmd  MOS) 

0.32 

10.22% 

0.58 

18.79% 

Component*Rater(Cmd  MOS) 

1.14 

36.46% 

1.57 

50.59% 

3.13 

100.00% 

xss«sasa 

3.11 

ssaaaaaa: 

100.00% 

Reliability 

Reliability 

Overall 

0.50 

0.30 

Within  Command 

0.52 

0.30 

Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  General  Soldiering 
(Officer  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.49 

47.47% 

0.81 

28.85% 

MOS 

0.00 

0.00% 

0.06 

2.06% 

Component*MOS 

0.07 

2.09% 

0.05 

1.88% 

Command 

0.00 

0.00% 

0.06 

1.98% 

Component*Command 

0.08 

2.41% 

0.00 

0.00% 

M0S*Command 

0.09 

2.74% 

0.00 

0.00% 

Rater(Cmd  MOS) 

0.27 

8.65% 

0.29 

10.37% 

Component*Rater(Cmd  MOS) 

1.15 

36.64% 

1.55 

54.86% 

3.14 

100.00% 

2.82 

100.00% 

Overall 

Within  Command 

Reliability 

0.50 

0.52 

Reliability 

0.33 

0.33 
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Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  General  Soldiering 
(FORSCOM  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.45 

42.34% 

0.69 

22.17% 

MOS 

0.04 

1.22% 

0.00 

0.00% 

Component*MOS 

0.38 

11.18% 

0.27 

8.81% 

Rank 

0.00 

0.00% 

0.06 

1.99% 

Component*Rank 

0.00 

0.00% 

0.02 

0.67% 

M0S*Rank 

0.01 

0.32% 

0.05 

1.56% 

Rater(Rnk  MOS) 

0.36 

10.42% 

0.48 

15.51% 

Component*Rater(Rnk  MOS) 

1.19 

34.53% 

1.53 

49.28% 

3.44 

a=333SSS 

100.00% 

33333333 

3.10 

S33SSS3S: 

100.00% 

Rel lability 

Rel lability 

Overal 1 

0.55 

0.31 

Within  Rank 

0.55 

0.32 

Phase  I  Test 


Variance  Components  for  Importance  Ratings  for  General  Soldiering 
(DOTD  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.34 

44.85% 

0.78 

28.15% 

MOS 

0.00 

0.00% 

0.02 

0.66% 

Component*MOS 

0.06 

2.10% 

0.00 

0.12% 

Rank 

0.04 

1.17% 

0.00 

0.00% 

Component*Rank 

0.00 

0.00% 

0.00 

0.00% 

M0S*Rank 

0.05 

1.52% 

0.02 

0.78% 

Rater(Rnk  MOS) 

0.22 

7.20% 

0.32 

11.72% 

Component*Rater(Rnk  MOS) 

1.29 

43.15% 

1.62 

58.58% 

2.99 

100.00% 

2.77 

100.00% 

Reliability  Reliability 


Overall 

0.47 

0.29 

Within  Rank 

0.48 

0.29 

Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  Overall  Job  Performance 
(NCO  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.19 

36.35% 

0.53 

17.14% 

MOS 

0.04 

1.15% 

0.00 

0.00% 

Component*MOS 

0.52 

15.82% 

0.54 

17.55% 

Command 

0.05 

1.49% 

0.00 

0.00% 

Component*Command 

0.00 

0.00% 

0.00 

0.00% 

MOS*Command 

0.00 

0.00% 

0.02 

0.63% 

Rater(Cd  M) 

0.30 

9.23% 

0.45 

14.50% 

Component*Rater(Cd 

1.17 

35.97% 

1.55 

50.18% 

3.26 

100.00% 

3.09 

100.00% 

Reliability  Reliability 

Overall  0.53  0.35 

Within  Command  0.54  0.35 


Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  Overall  Job  Performance 
(Officer  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.22 

42.79% 

0.45 

17.99% 

MOS 

0,02 

0.82% 

0.14 

5.71% 

Component*MOS 

0.44 

15.41% 

0.59 

23.47% 

Command 

0.00 

0.00% 

0.02 

0.78% 

Component*Command 

0.00 

0.00% 

0.04 

1.51% 

MOS*Command 

0.11 

3.98% 

0.00 

0.00% 

Rater(Cd  M) 

0.19 

6,76% 

0.06 

2.38% 

Component*Rater(Cd 

1  0.86 

30.23% 

1.20 

48.15% 

2.85 

100.00% 

2.49 

100.00% 

Rel iabil i 

ty 

Rel iabil i 

ty 

Overall 

0.59 

0.47 

Within  Command 

0.61 

0.48 
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Phase  I  Test 


Variance  Components  for  Importance  Ratings 

for  Overall 

Job  Performance 

(FORSCOM  ratings 

for  all  3  MOS) 

Task  Category 

Job  Activity 

Variance 

Percent 

Variance 

Percent 

Component 

1.18 

35.00% 

0.46 

14.95% 

MOS 

0.03 

1.03% 

0.00 

0.00% 

Component*MOS 

0.81 

24.02% 

0.67 

21.80% 

Rank 

0.00 

0.00% 

0.03 

0.98% 

Component*Rank 

0.01 

0.21% 

0.04 

1.16% 

MOS*Rank 

0.04 

1.11% 

0.13 

4.11% 

Rater(Rk  M) 

0.31 

9.27% 

0.43 

14.12% 

Component*Rater(Rk  0.99 

29.35% 

1.32 

42.88% 

3.36 

100.00% 

3.07 

100.00% 

Reliability 

Rel iabil ity 

Overall 

0.60 

0.37 

Within  Command 

0.61 

0.39 

Phase  I  Test 

Variance  Components  for  Importance  Ratings  for  Overall  Job  Performance 
(DOTD  ratings  for  all  3  MOS) 

Task  Category  Job  Activity 


Variance 

Percent 

Variance 

Percent 

Component 

1.19 

38.61% 

0.40 

14.89% 

MOS 

0.02 

0.50% 

0.08 

3.01% 

Component*MOS 

0.36 

11.77% 

0.53 

19.81% 

Rank 

0.04 

1.41% 

0.00 

0.00% 

Component*Rank 

0.02 

0.62% 

0.08 

2.87% 

M0S*Rank 

0.05 

1.56% 

0.02 

0.82% 

RaterfRk  M) 

0.18 

5.94% 

0.15 

5.52% 

Component*Rater(Rk  1.22 

39.59% 

1.42 

53.08% 

3.08 

100.00% 

2.68 

100.00% 

Reliability 

Reliability 

Overall 

0.51 

0.38 

Within  Command 

0.53 

0.39 

E-io  - 


Phase  I  Field  Test 

Summary  of  Single-rater  Reliability  Estimates  By  MOS 
for  Task  Category  and  Job  Activity  Importance  Ratings 
for  Core  Technical  Proficiency 


Task  Category 
IIB  63B  71L 


Job  Activity 
IIB  63B  71L 


All  Ratings 


Overall 

Within  Rank 

Within  Command 

Within  Rank  &  Command 

0.52 

0.52 

0.52 

0.54 

0.36 

0.37 

0.37 

0.41 

0.40 

0.41 

0.41 

0.42 

0.36 

0.36 

0.36 

0.40 

0.23 

0.24 

0.23 

0.25 

0.43 

0.44 

0.46 

0.47 

Pilot  Overall 

0.51 

0.59 

0.51 

0.40 

0.36 

0.43 

NCO  Ratings 

Overall 

Within  Command 

0.50 

0.50 

0.35 

0.37 

0.47 

0.48 

0.34 

0.35 

0.24 

0.25 

0.39 

0.40 

Pilot  Overall 

0.49 

0.57 

0.47 

0.38 

0.39 

0.39 

Officer  Ratings 

Overal 1 

Within  Command 

0.54 

0.55 

0.34 

0.36 

0.51 

0.51 

0.40 

0.41 

0.48 

0.48 

0.54 

0.55 

Pilot  Overall 

0.57 

0.63 

0.56 

0.49 

0.41 

0.46 

FORSCOM  Ratings 

Overal 1 

Within  Rank 

0.52 

0.53 

0.34 

0.36 

0.49 

0.49 

0.33 

0.36 

0.25 

0.27 

0.44 

0.44 

Pilot  Overall 

0.51 

0.56 

0.56 

0.40 

0.35 

0.44 

DOTO  Ratings 

Overal 1 

Within  Rank 

0.59 

0.59 

0.41 

0.43 

0.48 

0.48 

0.45 

0.45 

0.32 

0.35 

0.46 

0.47 

Pilot  Overall 

0.55 

0.64 

0.48 

0.42 

0.45 

0.41 

,1 
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Phase  I  Field  Test 

Summary  of  Single-rater  Reliability  Estimates 

for  Task  Category  and  Job  Activity  Importance  Ratings 

for  Overall  Job  Performance  by  MOS 


Task  Category 
118  638  71L 


Job  Activity 
118  638  71L 


All  Ratings 


Overall 

Within  Rank 

Within  Command 

Within  Rank  &  Command 

0.52 

0.52 

0.52 

0.52 

0.43 

0.44 

0.44 

0.46 

0.44 

0.45 

0.46 

0.48 

0.36 

0.36 

0.36 

0.39 

0.25 

0.26 

0.25 

0.27 

0.34 

0.34 

0.36 

0.36 

Pilot  Overall 

0.5i 

0.59 

0.51 

0.40 

0.36 

0.43 

NCO  Ratings 

Overall 

Within  Command 

0.49 

0.49 

0.40 

0.41 

0.49 

0.52 

0.33 

0.34 

0.27 

0.28 

0.33 

0.34 

Pilot  Overall 

0.49 

0.57 

0.47 

0.38 

0.39 

0.39 

Officer  Ratings 

Overall 

Within  Command 

0.55 

0.55 

0.52 

0.52 

0.48 

0.48 

0.39 

0.39 

0.30 

0.30 

0.38 

0.40 

Pilot  Overall 

0.57 

0.63 

0.56 

0.49 

0.41 

0.46 

FORSCOM  Ratings 

Overall 

Within  Rank 

0.52 

0.52 

0.44 

0.44 

0.49 

0.49 

0.33 

0.35 

0.24 

0.25 

0.35 

0.35 

Pilot  Overall 

0.51 

0.56 

0.56 

0.40 

0.35 

0.44 

GOTO  Ratings 

Overall 

Within  Rank 

0.55 

0.55 

0.44 

0.46 

0.51 

0.53 

0.40 

0.40 

0.27 

0.30 

0.39 

0.41 

Pilot  Overall 

0.55 

0.64 

0.48 

0.42 

0.45 

0.41 

E-12- 


Variance  Components  Tables  By  Each  MOS 


Task  Ratings  for: 

Core  Technical  Proficiency  (CTP) 
Overall  Job  Performance  (OJP) 

Activity  Ratings  for: 

CTP 

OJP 

Task  Ratings  for: 

CTP  for  NCOS 
CTP  for  Officers 
CTP  for  FORSCOM 
CTP  FOR  DOTD 

Activity  Ratings  for: 

CTP  for  NCOS 
CTP  for  Officers 
CTP  for  FORSCOM 
CTP  FOR  DOTD 

Task  Ratings  for: 

OJP  for  NCOS 
OJP  for  Officers 
OJP  for  FORSCOM 
OJP  FOR  DOTD 

Activity  Ratings  for: 

OJP  for  NCOS 
OJP  for  Officers 
OJP  for  FORSCOM 
OJP  FOR  DOTD 


E-IJ 


J 


Phase  I  Test 

Task  Category  Questionnaire 

Variance  Components  for  Importance  Ratings  for  Core  Technical  Proficiency 
(By  MOS) 


IIB 

63B 

71L 

Components 

Variance  Percent  Variance 

Percent  Variance 

Percent 

Task 

2.35 

51.77% 

1.09 

35.95% 

1.02 

39.89% 

Rank 

0.04 

0.89% 

0.00 

0.00% 

0.00 

0.14% 

Command 

0.01 

0.19% 

0.00 

0.01% 

0.00 

0.00% 

Rank  x  Command 

0.03 

0.70% 

0.23 

7.67% 

0.00 

0.00% 

Rater 

0.33 

7.30% 

0.26 

8.70% 

0.14 

5.57% 

Task  X  Rater 

1.67 

36.79% 

1.31 

43.27% 

1.25 

48.82% 

Task  X  Rank 

0.00 

0.00% 

0.09 

2.89% 

0.07 

2.68% 

Task  X  Command 

0.00 

0.00% 

0.05 

1.51% 

0.07 

2.90% 

Task  X  Rank  x  i 

Command  0.11 

2.36% 

0.00 

0.00% 

0.00 

0.00% 

Total 

4.55 

100.00% 

3.02 

100.00% 

2.56 

100.00% 

RELIABILITY 

IIB 

63B 

71L 

Overall 

0.52 

0.36 

0.40 

Within  Rank 

0.52 

0.37 

0.41 

Within  Command 

0.52 

0.37 

0.41 

Within  Rank  & 

Command  0.54 

0.41 

0.42 

E-14 


Phase  I  Test 

Task  Category  Questionnaire 

Variance  Components  for  Importance  Ratings  for  Overall  Job  Performance 
(By  MOS) 


IIB 

63B 

71L 

Components  ' 

i/ariance  Percent  Variance 

Percent  Variance 

Percent 

Task 

2.16 

51.54% 

1.49 

43.44% 

1.38 

44.08% 

Rank 

0.01 

0.26% 

0.00 

0.00% 

0.00 

0.00% 

Command 

0.00 

0.00% 

0.00 

0.00% 

0.00 

0.00% 

Rank  x  Command 

0.02 

0.43% 

0.09 

2.51% 

0.08 

2.43% 

Rater 

0.34 

8.00% 

0.35 

10.14% 

0.09 

2.94% 

Task  X  Rater 

1.63 

38.95% 

1.42 

41.57% 

1.40 

44.56% 

Task  X  Rank 

0.00 

0.00% 

0.01 

0.32% 

0.08 

2.55% 

Task  X  Command 

0.00 

0.00% 

0.07 

2.02% 

0.11 

3.45% 

Task  X  Rank  x  Command 

0.03 

0.81% 

0.00 

0.00% 

0.00 

0.00% 

Total 

4.19 

100.00% 

3.42 

100.00% 

3.13 

100.00% 

RELIABILITY 

IIB 

63B 

71L 

Overall 

0.52 

0.43 

0.44 

Within  Rank 

0.52 

0.44 

0.45 

Within  Command 

0.52 

0.44 

0.46 

Within  Rank  &  Command 

0.52 

0.46 

0.48 

e-15 


Phase  I  Test 

Job  Activity  Questionnaire 

Variance  Components  for  Importance  Ratings  for  Core  Technical  Proficiency 
(By  MOS) 


IIB  63B  71L 


Components  Variance 

Percent  Variance 

Percent  Variance 

Percent 

Activity 

1.45 

36.08% 

0.77 

22.90% 

1.29 

43.25% 

Rank 

0.00 

0.00% 

0.00 

0.02% 

0.00 

0.14% 

Command 

0.01 

0.13% 

0.04 

1.16% 

0.01 

0.50% 

Rank  x  Command 

0.36 

9.00% 

0.04 

1.08% 

0.00 

0.01% 

Rater 

0.22 

5.42% 

0.32 

9.37% 

0.33 

11.07% 

Activity  x  Rater 

1.98 

49.16% 

1.98 

58.81% 

1.10 

36.97% 

Activity  x  Rank 

0.00 

0.08% 

0.22 

6.47% 

0.08 

2.58% 

Activity  x  Command 

0.00 

0.00% 

0.01 

0.18% 

0.16 

5.48% 

Act.  X  Rank  x  Command 

0.01 

0.13% 

0.00 

0.00% 

0.00 

0.00% 

Total 

4.03 

100.00% 

3.37 

100.00% 

2.98 

100.00% 

RELIABILITY 

IIB 

63B 

71L 

Overall 

0.36 

0.23 

0.43 

Within  Rank 

0.36 

0.24 

0.44 

Within  Command 

0.36 

0.23 

0.46 

Within  Rank  &  Command 

0.40 

0.25 

0.47 
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Phase  I  Test 

Job  Activity  Questionnaire 

Variance  Components  for  Importance  Ratings  for  Overall  Job  Performance 
(By  MOS) 


IIB  63B  71L 


Components  Variance 

Percent  Variance 

Percent  Variance 

Percent 

Activity 

1.36 

35.86% 

0.66 

24.69% 

1.06 

33.88% 

Rank 

0.00 

0.00% 

0.04 

1.64% 

0.00 

0.03% 

Command 

0.00 

0.00% 

0.01 

0.48% 

0.01 

0.42% 

Rank  x  Command 

0.25 

6.62% 

0.00 

0.00% 

0.00 

0.00% 

Rater 

0.28 

7.39% 

0.26 

9.80% 

0.41 

13.01% 

Activity  x  Rater 

1.89 

49.76% 

1.57 

58.62% 

1.49 

47.60% 

Activity  x  Rank 

0.01 

0.37% 

0.09 

3.31% 

0.03 

0.80% 

Activity  x  Command 

0.00 

0.00% 

0.00 

0.00% 

0.13 

4.26% 

Act.  X  Rank  x  Command 

0.00 

0.00% 

0.04 

1.45% 

0.00 

0.00% 

Total 

3.80 

100.00% 

2.69 

100.00% 

3.12 

100.00% 

RELIABILITY 

IIB 

63B 

71L 

Overall 

0.36 

0.25 

0.34 

Within  Rank 

0.36 

0.26 

0.34 

Within  Command 

0.36 

0.25 

0.36 

Within  Rank  &  Command 

0.39 

0.27 

0.36 

/ 

r  * 
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Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(NCO  Ratings  By  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.10 

50.01% 

1.13 

35.48% 

1.03 

47.09% 

Command 

0.00 

0.00% 

0.11 

3.31% 

0.00 

0.00% 

Rater(Command) 

0.45 

10.73% 

0.51 

15.97% 

0.13 

5.75% 

Task*Rater 

1.65 

39.26% 

1.44 

45.24% 

1.00 

45.79% 

Task*Command 

0.00 

0.00% 

0.00 

0.00% 

0.03 

1.37% 

4.20  100.00%  3.18  100.00%  2.18  100.00% 


RELIABILITY 

Overall 

0.50 

0.35 

0.47 

Within  Command 

0.50 

0.37 

0.48 

Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(Officer  Ratings  By  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.56 

54.26% 

1.28 

34.38% 

1.16 

51.42% 

Command 

0.03 

0.69% 

0.12 

3.18% 

0.00 

0.06% 

Rater(Command) 

0.27 

5.70% 

0.23 

6.27% 

0.17 

7.40% 

Task*Rater 

1.86 

39.35% 

2.10 

56.18% 

0.93 

41.12% 

Task*Command 

0.00 

0,00% 

0.00 

0.00% 

0.00 

0.00% 

4.73 

100.00% 

3.74 

100.00% 

2.26 

100.00% 

RELIABILITY 

Overall 

0.54 

0.34 

0.51 

Within  Command 

0,55 

0.36 

0.51 
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Phase  I  Test;  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(FORSCOM  Ratings  By  MOS) 

liB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.29 

51.96% 

1.11 

33.83% 

1.12 

48.90% 

Rank 

0.07 

1.66% 

0.16 

4.76% 

0.00 

0.03% 

Rater{Rank) 

0.36 

8.09% 

0.47 

14.19% 

0.15 

6.71% 

Task*Rater 

1.69 

38.28% 

1.55 

47.23% 

1.02 

44.36% 

Task*Rank 

0.00 

0.00% 

0.00 

0.00% 

0.00 

0.00% 

4.41 

ssssasaas 

100.00% 

3.28 

aaaaaaaa 

100.00% 

2.30 

aaaaaaaa 

100.00% 

aaaaaaaaa 

RELIABILITY 


Overall 

0.52 

0.34 

0.49 

Within  Rank 

0.53 

0.36 

0.49 

Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(DOTD  Ratings  By  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.46 

59.02% 

1.22 

41.12% 

1.03 

48.05% 

Rank 

0.00 

0.01% 

0.07 

2.52% 

0.01 

0.57% 

Rater(Rank) 

0.25 

5.91% 

0.29 

9.68% 

0.13 

5.99% 

Task*Rater 

1.46 

35.06% 

1.35 

45.61% 

0.97 

45.40% 

Task*Rank 

0.00 

0.00% 

0.03 

1.07% 

0.00 

0.00% 

4.16 

100.00% 

2.97 

100.00% 

2.15 

100.00% 

RELIABILITY 


Overall  0.59 

Within  Rank  0.59 


0.41  0.48 

0.43  0.48 


Phase  I  Test:  Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(NCO  Ratings  By  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Act 

1.33 

33.62% 

0.70 

24.09% 

1.21 

38.94% 

Command 

0.12 

3.00% 

0.10 

3.51% 

0.01 

0.24% 

Rater(Command) 

0.53 

13.31% 

0.41 

14.12% 

0.53 

17.08% 

Act*Rater 

1.99 

50.07% 

1.68 

57.68% 

1.26 

40.79% 

Act*Command 

0.00 

0.00% 

0.02 

0.60% 

0.09 

2.95% 

3.97 

100.00% 

2.92 

100.00% 

3.10 

100.00% 

RELIABILITY 

Overall  0.34 

Within  Command  0.35 


0.24  0.39 

0.25  0.40 


Phase  I  Test:  Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(Officer  Ratings  By  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Act 

1.40 

40.14% 

1.51 

47.79% 

1.59 

53.85% 

Command 

0.08 

2.29% 

0.00 

0.03% 

0.02 

0.64% 

Rater(Command) 

0.17 

4.97% 

0.22 

7.08% 

0.16 

5.52% 

Act*Rater 

1.84 

52.61% 

1.43 

45.09% 

1.11 

37.73% 

Act*Command 

0.00 

0.00% 

0.00 

0.00% 

0.07 

2.25% 

3.50  100.00%  3.16  100.00%  2.95  100.00% 


RELIABILITY 


Overall 

Within  Command 


0.40 

0.41 


0.48 

0.48 


0.54 

0.55 


Phase  I  Test:  Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical  Proficiency 
(FORSCOM  Ratings  By  MOS) 


IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Act 

1.21 

33.11% 

0.73 

24.67% 

1.56 

44.20% 

Rank 

0.31 

8.40% 

0.00 

0.00% 

0.00 

0.00% 

Rater(Rank) 

0.30 

8.18% 

0.46 

15.44% 

0.42 

12.00% 

Act*Rater 

1.83 

50.05% 

1.56 

52.68% 

1.54 

43.79% 

Act*Rank 

0.01 

0.26% 

0.21 

7.21% 

0.00 

0.00% 

3.66 

sssssasMs 

100.00% 

aaaxatasaaa 

2.97 

aaaaaaaa 

100.00% 

aaaaaaaaaaaa 

3.52 

100.00% 

RELIABILITY 


Overall 

0.33 

0.25 

0.44 

Within  Rank 

0.36 

0.27 

0.44 

Phase  I  Test: 

Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Core  Technical 

Proficiency 

(GOTO  Ratings  By  MOS) 

IIB 

63B 

71L 

Variance 

Percent  Variance  Percent  Variance 

Percent 

Act 

1.95 

44.82% 

0.98 

31.92% 

1.21 

45.80% 

Rank 

0.00 

0.00% 

0.09 

2.94% 

0.00 

0.00% 

Rater(Rank) 

0.43 

9.96% 

0.19 

6.26% 

0.22 

8.39% 

Act*Rater 

1.97 

45.21% 

1.62 

52.62% 

1.13 

43.12% 

Act*Rank 

0.00 

0.00% 

0.19 

6.26% 

0.07 

2.69% 

4.36 

100.00% 

3.09 

100.00% 

2.63 

100.00% 

RELIABILITY 

Overall 

0.45 

0.32 

0.46 

Within  Rank 

0.45 

0.35 

0.47 

E-21 


Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 

(NCO  Ratings  By  MOS) 


IIB  63B  71L 


Variance 

Percent  Vari 

ance 

Percent  Variance 

Percent 

Task 

2.06 

49.13% 

1.43 

40.37% 

1.55 

48.80% 

Command 

0.00 

0.00% 

0.06 

1.76% 

0.05 

1.64% 

Rater(Command) 

0.39 

9.23% 

0.48 

13.60% 

0.09 

2.87% 

Task*Rater 

1.75 

41.64% 

1.57 

44.27% 

1.35 

42.29% 

Task*Command 

0.00 

0.00% 

0.00 

0.00% 

0.14 

4.39% 

4.19  100.00%  3.54  100.00%  3.18  100.00% 


RELIABILITY 


Overall 

0.49 

0.40 

0.49 

Within  Command 

0.49 

0.41 

0.52 

Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 

(Officer  Ratings  By  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.25 

54.66% 

1.64 

51.61% 

1.39 

47.61% 

Command 

0.01 

0.29% 

0.03 

0.91% 

0.04 

1.21% 

Rater(Command) 

0.31 

7.46% 

0,27 

8.39% 

0.17 

5.67% 

Task*Rater 

1.55 

37.59% 

1.24 

39.08% 

1.33 

45.51% 

Task*Command 

0.00 

0,00% 

0.00 

0.00% 

0.00 

0.00% 

4.12 

100.00% 

3,17 

100.00% 

2.91 

100.00% 

RELIABILITY 

Overall 

0.55 

0.52 

0.48 

Within  Command 

0.55 

0.52 

0.48 
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Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 

(FORSCOM  Ratings  By  MOS) 


IIB 


63B 


71L 


Variance 

Percent  Variance 

Percent  Vari 

ance 

Percent 

Task 

2.12 

51.69% 

1.54 

43.71% 

1.49 

49.00% 

Rank 

0.03 

0.71% 

0.01 

0.26% 

0.02 

0.65% 

Rater(Rank) 

0.35 

8.52% 

0.47 

13.31% 

0.17 

5.56% 

Task*Rater 

1.60 

39.08% 

1.51 

42.72% 

1.37 

44.79% 

Task*Rank 

0.00 

0.00% 

0.00 

0.00% 

0.00 

0.00% 

4.10  100.00%  3.53  100.00%  3.05  100.00% 


ssssssssBSssaaasssssasssasssassssssaaaassasaa&sssssssssss 


RELIABILITY 


Overall 

0.52 

0.44 

0.49 

Within  Rank 

0.52 

0.44 

0.49 

Phase  I  Test:  Task  Category  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 

(GOTO  Ratings  By  MOS) 


IIB  638  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Task 

2.26 

55.05% 

1.55 

44.04% 

1.48 

51.47% 

Rank 

0.00 

0.00% 

0.11 

3.04% 

0.08 

2.64% 

Rater(Rank) 

0,25 

6.11% 

0.28 

8.06% 

0.07 

2.28% 

Task*Rater 

1.59 

38.84% 

1.56 

44.11% 

1.26 

43.60% 

Task*Rank 

0.00 

0.00% 

0.03 

0.74% 

0.00 

0.00% 

4.10 

100.00% 

3.53 

100.00% 

2.88 

100.00% 

RELIABILITY 


Overall 

0.55 

0.44 

0.51 

Within  Rank 

0.55 

0.46 

0.53 

E-23 


Phase  I  Test:  Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 

(NCO  Ratings  by  MOS) 


IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Act 

1.31 

33.15% 

0.74 

27.02% 

1.10 

32.79% 

Command 

0.11 

2.77% 

0.01 

0.35% 

0.01 

0.31% 

Rater(Command) 

0.53 

13.41% 

0.27 

9.83% 

0.50 

15.10% 

Act*Rater 

2.01 

50.68% 

1.66 

60.44% 

1.66 

49.69% 

Act*Command 

0.00 

0.00% 

0.06 

2.36% 

0.07 

2.11% 

3.97  100.00%  2.75  100.00%  3.34  100.00% 


Overall 

0.33 

RELIABILITY 

0.27 

0.33 

Within  Command 

0.34 

0.28 

0.34 

Phase  I  Test: 

Job  Activity  Questionnaire 

Variance  Components  for  : 

Importance  for  Overall  Job  Performance 

(Officer  Ratings  by  MOS) 

IIB 

63B 

71L 

Variance 

Percent  Variance  Percent  Variance  Percent 

Act 

1.28 

39.19% 

0.80  30.22% 

1.13 

38.46% 

Command 

0.02 

0.47% 

0.01  0.50% 

0.01 

0.35% 

Rater(Command) 

0.22 

6.61% 

0.33  12.28% 

0.34 

11.47% 

Act*Rater 

1.75 

53.73% 

1.52  57.01% 

1.37 

46.76% 

Act*Command 

0.00 

0.00% 

0.00  0.00% 

0.09 

2.96% 

3.26 

100.00% 

2.66  100.00% 

2.93 

100.00% 

RELIABILITY 

Overall 

0.39 

0.30 

0.38 

Within  Command 

0.39 

0.30 

0.40 

E-24 


Phase  I  Test:  Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 
(FORSCOM  Ratings  by  MOS) 


IIB  63B  71L 

Variance  Percent  Variance  Percent  Variance  Percent 


Act 

1.19 

32.76% 

0.60 

24.02% 

1.23 

34.93% 

Rank 

0.21 

5.69% 

0.04 

1.79% 

0.00 

0.00% 

Rater(Rank) 

0.33 

9.18% 

0.26 

10.55% 

0.53 

15.10% 

Act*Rater 

1.89 

51.97% 

1.49 

59.83% 

1.75 

49.98% 

Act*Rank 

0.01 

0.39% 

0.09 

3.81% 

0.00 

0.00% 

3.63 

100.00% 

aaassssass 

2.49 

100.00% 

asaaaaaaaa 

3.51 

aaaasaa 

100.00% 

RELIABILITY 


Overall 

0.33 

0.24 

0.35 

Within  Rank 

0.35 

0.25 

0.35 

Phase  I  Test:  Job  Activity  Questionnaire 

Variance  Components  for  Importance  for  Overall  Job  Performance 
(DOTD  Ratings  by  MOS) 

IIB  63B  71L 


Variance 

Percent  Variance 

Percent  Variance 

Percent 

Act 

1.75 

39.68% 

0.82 

27.33% 

1.10 

39.36% 

Rank 

0.00 

0.00% 

0.04 

1.28% 

0.00 

0.00% 

Rater(Rank) 

0.52 

11.82% 

0.30 

10.00% 

0.24 

8.47% 

Act*Rater 

2.14 

48.50% 

1.65 

55.03% 

1.36 

48.60% 

Act*Rank 

0.00 

0.00% 

0.19 

6.36% 

0.10 

3.56% 

4.41 

100.00% 

3.00 

100.00% 

2.80 

100.00% 

RELIABILITY 

Overall 

0.40 

0.27 

0.39 

Within  Rank 

0.40 

0.30 

0.41 
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Phase  I  Test 

Mean  Fidelity  Coefficients  for  4  Ratings  with 
Frequency  of  Performance 


Task  Category  Job  Activity 


MOS 

Rating  Type  i 

IIB 

(n=88) 

63B 

(n=50) 

71L 

{n=52) 

IIB 

(n=88) 

63B 

{n=50) 

71L 

{n=52) 

Frequency 

0.76 

0.47 

0.28 

0.67 

0.30 

0.21 

Core  Technical 

0.75 

0.44 

0.25 

0.66 

0.27 

0.18 

IIB 

Gen.  Soldiering 

0.75 

0.46 

0.28 

0.66 

0.28 

0.25 

Overall 

0.75 

0.46 

0.26 

0.66 

0.27 

0.23 

Frequency 

0.49 

0.74 

0.35 

0.33 

0.62 

0.05 

Core  Technical 

0.43 

0.73 

0.35 

0.21 

0.59 

-0.02 

63B 

Gen.  Soldiering 

0.67 

0.66 

0.38 

0.57 

0.49 

0.17 

Overall 

0.59 

0.72 

0.36 

0.41 

0.60 

0.05 

Frequency 

0.27 

0.33 

0.79 

0.19 

0.03 

0.72 

Core  Technical 

0.18 

0.27 

0.77 

0.10 

-0.02 

0.69 

71L 

Gen.  Soldiering 

0.63 

0.48 

0.56 

0.49 

0.19 

0.58 

Overall 

0.47 

0.41 

0,73 

0.31 

0.06 

0.69 

Phase  I  Test 

Mean  Fidelity  Coefficients  for  4  Ratings  with 
Importance  for  Core  Technical  Proficiency 


Task  Category  Job  Activity 

IIB  63B  71L  IIB  63B  71L 


MOS 

Rating  Type  (n-88) 

(n=50) 

{n-52) 

(n=87) 

(n-50) 

(n=51) 

Frequency 

0.77 

0.38 

0.17 

0.64 

0.19 

0.11 

Core  Technical 

0.77 

0.35 

0.14 

0.65 

0.16 

0.09 

IIB 

Gen.  Soldiering 

0.77 

0.37 

0.17 

0.64 

0.18 

0.17 

Overall 

0.77 

0.37 

0.15 

0.65 

0.17 

0.14 

Frequency 

0.47 

0.67 

0.28 

0.28 

0.56 

-0.02 

Core  Technical 

0.41 

0.69 

0.29 

0.16 

0.59 

-0.04 

63B 

Gen.  Soldiering 

0.67 

0.57 

0.27 

0.55 

0.39 

0.08 

Overall 

0.58 

0.64 

0.26 

0.37 

0.54 

-0.03 

Frequency 

0.25 

0.31 

0.73 

0.16 

-0.01 

0.71 

Core  Technical 

0.15 

0.27 

0.75 

0.07 

-0.02 

0.74 

71L 

Gen.  Soldiering 

0.63 

0.40 

0.44 

0.46 

0.10 

0.53 

Overall 

0.46 

0.36 

0.64 

0.29 

0.00 

0.68 
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Phase  I  Test 

Mean  Fidelity  Coefficients  for  4  Ratings  with 
Importance  for  General  Soldiering 

Task  Category  Job  Activity 


MOS 

118 

Rating  Type  {n=88) 

638 

(n=50) 

71L 

(n=52) 

118 

(n=87) 

638 

{n=50) 

71L 

(n=50) 

Frequency 

0.74 

0.64 

0.60 

0.61 

0.47 

0.45 

Core  Technical 

0.75 

0.62 

0.59 

0.62 

0.47 

0.45 

118 

Gen.  Soldiering 

0.75 

0.64 

0.61 

0.62 

0.48 

0.49 

Overall 

0.75 

0.63 

0.60 

0.62 

0.47 

0.47 

Frequency 

0.47 

0.65 

0.47 

0.27 

0.42 

0.19 

Core  Technical 

0.41 

0.60 

0.43 

0.17 

0.35 

0.09 

638 

Gen.  Soldiering 

0.66 

0.73 

0.64 

0.54 

0.55 

0.41 

Overall 

0.58 

0.71 

0.57 

0.36 

0.48 

0.24 

Frequency 

0.27 

0.36 

0.52 

0.21 

0.14 

0.50 

Core  Technical 

0.17 

0.27 

0.43 

0.14 

0.07 

0.45 

7IL 

Gen.  Soldiering 

0.63 

0.64 

0.72 

0.49 

0.37 

0.62 

Overal 1 

0.47 

0.52 

0.66 

0.33 

0.22 

0.57 

Phase  I  Test 

Mean  Fidelity  Coefficients  for  4  Ratings  with 

Importance  for  Overall  Performance 

Task  Category  Job  Activity 

118  638  71L  118  638  71L 


MOS 

Rating  Type  (n-88) 

(n»50) 

(n-52) 

(n-87) 

{n*50) 

(n-51) 

Frequency 

0.76 

0.57 

0.46 

0.64 

0.34 

0.31 

Core  Technical 

0.76 

0.55 

0.44 

0.65 

0.33 

0.30 

118 

Gen,  Soldiering 

0.76 

0.56 

0.47 

0.65 

0.33 

0.36 

Overall 

0.77 

0.56 

0.45 

0.65 

0.33 

0.34 

Frequency 

0.48 

0.72 

0.41 

0.28 

0.55 

0.07 

Core  Technical 

0.42 

0.69 

0.39 

0.17 

0.52 

-0.01 

638 

Gen.  Soldiering 

0.67 

0.71 

0.53 

0.55 

0.51 

0.26 

Overall 

0.59 

0.74 

0.48 

0.37 

0.57 

0.10 

Frequency 

0.26 

0.34 

0.69 

0.20 

0.04 

0.65 

Core  Technical 

0.16 

0.26 

0.63 

0.12 

-0.02 

0.61 

71L 

Gen.  Soldiering 

0.63 

0.57 

0.68 

0.49 

0.22 

0.62 

Overall 

0.47 

0.47 

0.74 

0.32 

0.08 

0.67 

) 


E-27 


Tables  of  Component  Rating  Means,  Standard  Deviations,  and  N 
Task  Category  Instrument 

lie 

638 

71L 

All  3  MOS  combined 

Job  Activity  Instrument 
IIB 
638 
7IL 

All  3  MOS  combined 


E-28 


TASK  CATEGORY 

Frequency  and  Inportance  Ratings  Descriptive  Statistics 


Frequency 

Core  Technical 

Gen.  Soldiering 

Overall  Job 

Task  Categories 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Perf  operator  maint  chks  and  services 

3.75 

1.42 

88 

3.89 

1.41 

88 

3.56 

1.46 

87 

3.87 

1.37 

86 

Perf  operator  chks  &  services  on  weapons 

4.2A 

0.90 

88 

4.50 

0.86 

88 

4.19 

1.12 

88 

4.41 

0.87 

88 

Troubleshoot  mechanical  systems 

1.72 

1.75 

88 

1.92 

1.84 

88 

1.74 

1.73 

88 

1.93 

1.88 

88 

Repair  weapons 

1.40 

1.38 

87 

1.86 

1.75 

87 

1.69 

1.68 

87 

1.85 

1.76 

87 

Repair  mechanical  systems 

1.64 

1.61 

88 

1.80 

1.73 

88 

1.80 

1.73 

88 

1.93 

1.83 

88 

Troubleshoot  weapons 

2.26 

1.62 

88 

2.82 

1.82 

88 

2.51 

1.72 

88 

2.65 

1.71 

88 

Install  electronic  confxsnents 

2.33 

1.72 

88 

2.45 

1.84 

88 

2.33 

1.78 

88 

2.35 

1.81 

88 

Inspect  electrical  systems 

0.49 

1.11 

88 

0.57 

1.30 

87 

0.52 

1.15 

87 

0.52 

1.15 

87 

Inspect  electronic  systems 

0.57 

1.15 

88 

0.61 

1.27 

87 

0.52 

1.09 

87 

0.55 

1.18 

87 

Repair  electrical  systems 

0.45 

0.99 

88 

0.47 

1.08 

87 

0.48 

1.06 

87 

0.54 

1.15 

87 

Repair  electronic  components 

0.39 

0.94 

88 

0.40 

1.02 

88 

0.41 

1.00 

88 

0.42 

0.98 

88 

Pack  and  load  materials 

2.03 

1.58 

88 

2.10 

1.58 

87 

1.93 

1.52 

87 

1.95 

1.55 

87 

Prepare  parachutes 

0.34 

0.94 

87 

0.36 

1.00 

87 

0.37 

0.98 

87 

0.37 

1.00 

87 

Prepare  equip  and  supplies  for  air  drop 

0.57 

0.99 

88 

0.69 

1.30 

88 

0.72 

1.33 

88 

0.72 

1.35 

88 

Operate  power  excavating  eouioment 

0.17 

0.61 

87 

0.16 

0.55 

87 

0.14 

0.55 

87 

0.16 

0.59 

87 

Operate  wheeled  vehicles 

2.53 

1.52 

88 

2.68 

1.43 

87 

2.57 

1.48 

87 

2.74 

1.53 

87 

Operate  track  vehicles 

2.74 

2.01 

88 

2.77 

2.02 

88 

2.43 

1.85 

88 

2.67 

1.95 

88 

Operate  boats 

0.45 

0.92 

88 

0.60 

1.19 

88 

0.49 

1.05 

88 

0.52 

1.02 

88 

Operate  lifting, loading, Sgrading  equip 

0.15 

0.67 

88 

0.11 

0.56 

87 

0.10 

0.48 

87 

0.10 

0.48 

87 

Paint 

1.70 

1.23 

88 

1.07 

1.20 

86 

1.27 

1.22 

86 

1.22 

1.18 

86 

Install  wire  and  cables 

1.52 

1.49 

88 

1.84 

1.70 

86 

1.65 

1.62 

85 

1.66 

1.64 

85 

Repair  plastic  arxj  fiberglass 

0.22 

0.73 

88 

0.16 

0.57 

88 

0.18 

0.60 

88 

0.13 

0.42 

88 

Repair  metal 

0.23 

0.66 

88 

0.16 

0.52 

88 

0.20 

0.65 

88 

0.24 

0.69 

88 

Assemble  steel  structures 

0.22 

0.69 

88 

0.26 

0.80 

88 

0.26 

0.80 

88 

0.27 

0.81 

88 

Install  pipe  assemblies 

0.13 

0.60 

88 

0.15 

0.67 

88 

0.14 

0.59 

88 

0.15 

0.67 

88 

Construct  wooden  bldgs  &  other  struct 

0.28 

0.79 

88 

0.24 

0.75 

87 

0.29 

0.78 

87 

0.28 

0.80 

87 

Construct  masonry  bldgs  (  structures 

0.10 

0.50 

88 

0.10 

0.50 

88 

0.10 

0.50 

88 

0.11 

0.60 

88 

Operate  gas  1  electric  powered  equip 

0.64 

0.98 

88 

0.60 

1.01 

88 

0.61 

1.02 

88 

0.64 

1.08 

88 

Select, layout, (clean  med/dental  equip 

0.07 

0.45 

88 

0.06 

0.35 

88 

0.07 

0.45 

88 

0.07 

0.45 

88 

Use  audiovisual  equipment 

0.63 

0.97 

88 

0.66 

1.18 

87 

0.72 

1.16 

87 

0.71 

1.18 

87 

Reproduce  printed  material 

0.86 

1.27 

88 

0.53 

0.95 

87 

0.74 

1.19 

87 

0.70 

1.11 

87 

Operate  electronic  equipment 

1.27 

1.64 

88 

1.41 

1.76 

88 

1.26 

1.64 

88 

1.36 

1.78 

87 

Operate  radar 

0.13 

0.64 

88 

0.10 

0.50 

88 

0.11 

0.53 

88 

0.11 

0.60 

88 

Operate  computer  hardware 

0.40 

0.85 

88 

0.35 

0.88 

88 

0.47 

1.04 

88 

0.50 

1.13 

88 

Cook 

0.40 

1.13 

87 

0.36 

1.05 

87 

0.38 

1.11 

87 

0.39 

1.07 

87 

Perform  medical  laboratory  procedures 

0.18 

0.77 

88 

0.18 

0.81 

88 

0.17 

0.75 

88 

0.18 

0.81 

88 

Conduct  land  surveys 

0.94 

1.65 

87 

1.02 

1.76 

86 

0.87 

1.56 

86 

0.90 

1.59 

86 

Provide  medical  or  dental  treatment 

0.26 

0.84 

87 

0.33 

0.96 

86 

0.34 

1.01 

86 

0.30 

0.98 

86 

Sketch  maps,  overlays,  or  range  cards 

3.33 

1.45 

88 

3.61 

1.53 

87 

3.09 

1.59 

87 

3.41 

1.55 

87 

Produce  technical  drawings 

0.15 

0.60 

88 

0.17 

0.68 

88 

0.17 

0.73 

88 

0.15 

0.65 

88 

Draw  maps  and  overlays 

0.89 

1.43 

88 

1.13 

1.71 

88 

,0.95 

1.50 

88 

0.98 

1.52 

88 

Draw  illustrations 

0.56 

1.13 

88 

0.66 

1.35 

87 

0.61 

1.25 

87 

0.54 

1.17 

87 

Type 

0.70 

1.10 

88 

0.55 

0.99 

88 

0.68 

1.23 

87 

0.52 

0.97 

88 

Prepare  technical  forms  and  docunents 

1.03 

1.41 

88 

0.99 

1.53 

87 

1.02 

1.52 

87 

0.95 

1.45 

87 

Record,  file,  and  dispatch  information 

0.91 

1.28 

88 

0.80 

1.27 

87 

0.90 

1.35 

87 

0.89 

1.34 

87 

Receive, store, (issue  sibp,  equip,  etc 

0.91 

1.37 

88 

0.92 

1.40 

87 

0.94 

1.43 

87 

1.01 

1.47 

87 

Use  hand  and  arm  signals 

4.10 

0.98 

88 

4.28 

1.10 

87 

3.82 

1.28 

87 

4.05 

1.12 

87 

Read  tech/field  manuals,  reg  etc 

3.67 

1.10 

88 

3.86 

1.14 

87 

3.80 

1.18 

87 

3.85 

1.14 

87 
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TASK  CATEGORY  (Contd) 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


H0S=11B 


Frequency  Core  Technical  Gen.  Soldiering  Overall  Job 


Task  Categories 

MEAN 

STD 

H 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Use  maps 

4.02 

1.11 

88 

4.45 

0.82 

87 

3.98 

1.13 

87 

4.10 

1.07 

87 

Send  and  receive  radio  messages 

3.58 

1.14 

88 

4.20 

0.96 

87 

3.78 

1.14 

87 

3.90 

1.05 

87 

Give  short  oral  reports 

3.77 

1.14 

88 

4.21 

1.08 

87 

3.74 

1.32 

87 

3.97 

1.09 

87 

Receive  clients,  patients,  guests 

0.32 

0.89 

88 

0.28 

0.84 

87 

0.30 

0.85 

87 

0.31 

0.89 

87 

Give  directions  and  instructions 

2.78 

1.56 

87 

3.07 

1.66 

86 

2.87 

1.68 

86 

2.99 

1.70 

£6 

Write  and  deliver  presentations 

0.58 

1.17 

88 

0.66 

1.40 

87 

0.61 

1.24 

87 

0.64 

1.34 

87 

Interview 

0.53 

1.18 

88 

0.61 

1.31 

88 

0.60 

1.32 

88 

0.61 

1.31 

88 

Provide  coinseling 

1.18 

1.63 

88 

1.28 

1.78 

87 

1.26 

1.70 

87 

1.32 

1.86 

87 

Write  documents  and  correspondence 

0.67 

1.25 

88 

0.80 

1.52 

88 

0.78 

1.53 

88 

0.70 

1.37 

88 

Decode  data 

2.15 

1.44 

88 

2.80 

1.70 

86 

2.53 

1.61 

86 

2.52 

1.66 

85 

Analyze  electronic  signals 

0.33 

0.88 

88 

0.50 

1.22 

88 

0.41 

1.02 

88 

0.43 

1.09 

88 

Analyze  weather  conditions 

0.90 

1.41 

88 

1.14 

1.82 

87 

0.90 

1.49 

87 

0.97 

1.62 

87 

Order  equipment  and  supplies 

1.02 

1.40 

88 

1.27 

1.74 

86 

1.15 

1.62 

86 

1.13 

1.60 

86 

Estimate  time  t  cost  of  maint  ops 

0.55 

1.41 

88 

0.45 

1.28 

86 

0.34 

1.11 

85 

0.41 

1.18 

85 

Plan  placement/use  of  tactical  equip 

1.78 

1.85 

88 

2.12 

2.08 

86 

1.77 

1.85 

86 

1.99 

1.94 

86 

Translate  foreign  languages 

0.25 

0.86 

88 

0.21 

0.78 

87 

0.26 

0.96 

87 

0.22 

0.78 

87 

Analyze  intelligence  data 

1.13 

1.53 

88 

1.40 

1.76 

86 

1.22 

1.59 

86 

1.28 

1.64 

86 

Control  money 

0.78 

1.44 

88 

0.80 

1.58 

87 

0.82 

1.57 

87 

0.86 

1.62 

87 

Determine  firing  data* indirect  weapons 

1.05 

1.55 

88 

1.24 

1.79 

86 

1.05 

1.60 

86 

1.14 

1.70 

86 

Compute  statistics/other  math 

0.48 

1.05 

88 

0.52 

1.21 

86 

0.52 

1.15 

86 

0.52 

1.17 

86 

Provide  progranming  t  DP  sv^sport 

0.22 

0.65 

88 

0.26 

0.91 

86 

0.23 

0.79 

86 

0.17 

0.62 

86 

Control  air  traffic 

0.16 

0.57 

88 

0.25 

0.86 

88 

0.18 

0.64 

88 

0.20 

0.76 

88 

Use  hand  grenades 

3.38 

1.23 

88 

4.22 

1.07 

86 

3.80 

1.28 

86 

4.06 

1.06 

86 

Protect  against  NBC  hazards 

3.81 

1.10 

88 

4.63 

0.55 

86 

4.35 

0.88 

86 

4.42 

0.87 

86 

Handle  demolitions  or  mines 

2.42 

1.53 

88 

3.34 

1.72 

87 

2.83 

1.67 

87 

3.07 

1.70 

87 

Engage  in  hand-to'hand  combat 

2.31 

1.49 

88 

3.38 

1.66 

87 

3.07 

1.67 

87 

3.22 

1.57 

87 

Fire  individual  weapons 

4.19 

1.02 

88 

4.67 

0.56 

87 

4.44 

0.96 

87 

4.52 

0.76 

87 

Control  individuals  and  crowds 

2.97 

1.41 

88 

3.34 

1.50 

87 

3.11 

1.60 

87 

3.38 

1.47 

87 

Customs  and  laws  of  war 

3.03 

1.43 

88 

3.54 

1.44 

85 

3.48 

1.36 

86 

3.47 

1.39 

86 

Navi  gate 

3.68 

1.27 

88 

4.30 

1.06 

87 

3.90 

1.29 

87 

4.05 

1.17 

87 

Survive  in  the  field 

4.05 

1.29 

88 

4.33 

1.14 

87 

3.98 

1.39 

86 

4.16 

1.25 

87 

Move  and  react  in  the  field 

4.27 

1.00 

88 

4.60 

0.72 

87 

4.06 

1.21 

87 

4.38 

0.93 

87 

Load  (  unload  field  artillery/ tank  gens 

0.62 

1.39 

87 

0.71 

1.58 

87 

0.60 

1.35 

87 

0.68 

1.50 

87 

Fire  heavy  direct  fire  weapons 

1.58 

1.80 

88 

2.03 

2.12 

88 

1.65 

1.89 

88 

1.86 

2.02 

88 

Prepare  heavy  weapons  for  tactical  use 

0.60 

1.37 

88 

0.70 

1.57 

88 

0.60 

1.39 

88 

0.63 

1.45 

88 

Place  (  camoufl  tactical  equip  and  mat 

2.97 

1.73 

88 

3.51 

1.74 

88 

3.10 

1.67 

88 

3.38 

1.72 

88 

Fire  indirect  fire  weapons 

0.81 

1.53 

88 

0.84 

1.63 

87 

0.86 

1.66 

87 

0.85 

1.64 

87 

Give  first  aid 

3.41 

1.17 

88 

4.35 

0.96 

86 

4.28 

0.93 

86 

4.31 

0.80 

86 

Detect  and  identify  targets 

3.64 

1.19 

88 

4.28 

0.95 

87 

3.86 

1.18 

87 

4.05 

1.07 

87 

Plan  operations 

1.67 

1.74 

88 

2.22 

2.09 

87 

2.06 

2.01 

87 

2.11 

2.05 

87 

Direct/ lead  teams 

2.10 

1.82 

88 

2.70 

2.10 

87 

2.28 

1.93 

87 

2.41 

2.02 

87 

Honi tor/inspect 

2.35 

1.88 

88 

2.69 

1.98 

87 

2.57 

1.94 

87 

2.68 

2.00 

87 

Lead 

2.56 

1.7V 

88 

3.00 

1.99 

86 

2.84 

1.96 

86 

3.02 

1.97 

86 

Act  as  a  awdel 

2.78 

1.80 

88 

3.15 

1.95 

86 

3.09 

1.90 

86 

3.17 

1.93 

86 

Counsel 

2.11 

1.88 

88 

2.53 

2.07 

87 

2.46 

2.04 

87 

2.53 

2.10 

87 

Conrnuricate 

2.75 

1.95 

88 

3.14 

2.04 

87 

2.91 

2.02 

87 

3.07 

2.03 

86 

Train 

2.03 

1.96 

88 

2.45 

2.18 

87 

2.33 

2.09 

87 

2.41 

2.17 

87 

Personnel  A^instration 

1.58 

1.94 

88 

1.66 

2.02 

87 

1.59 

1.96 

87 

1.64 

2.01 

87 
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TASK  CATEGORY 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


HOS=638 


Frequency 

Task  Categories 

HEAN 

STO 

Perf  operator  mint  chks  and  services 

4.02 

1.06 

Perf  operator  chks  t  services  on  weapons 

2.64 

1.14 

Troubleshoot  mechanical  systems 

3.66 

1.27 

Repair  weapons 

0.84 

1.27 

Repair  mechanical  systems 

4.12 

0.94 

Troubleshoot  weapons 

0.90 

1.28 

Install  electronic  components 

1.36 

1.35 

Inspect  electrical  systems 

3.38 

1.23 

Inspect  electronic  systems 

1.34 

1.70 

Repair  electrical  systems 

2.96 

1.26 

Repair  electronic  components 

1.78 

1.61 

Pack  and  load  mterials 

1.68 

1.33 

Prepare  parachutes 

0.06 

0.42 

Prepare  equip  and  supplies  for  air  drop 

0.24 

0.56 

Operate  power  excavating  equipment 

0.26 

0.63 

Operate  wheeled  vehicles 

3.92 

0.94 

Operate  track  vehicles 

1.40 

1.43 

Operate  boats 

0.06 

0.31 

Operate  lifting, loading, (grading  equip 

1.42 

1.36 

Paint 

1.66 

1.32 

Install  wire  and  cables 

1.20 

1.16 

Repair  plastic  and  fiberglass 

0.66 

0.89 

Repair  metal 

1.56 

1.09 

Assetible  steel  structures 

0.20 

0.57 

Install  pipe  assemblies 

0.18 

0.44 

Construct  wooden  bldgs  t  other  struct 

0.18 

0.44 

Construct  msonry  bldgs  t  structures 

0.06 

0.31 

Operate  gas  (  electric  powered  equip 

2.78 

1.27 

Select, layout, (clean  med/dental  equip 

0.06 

0.31 

Use  audiovisual  equipment 

0.40 

0.83 

Reproduce  printed  raterial 

0.80 

1.14 

Operate  electronic  equipment 

0.92 

1.12 

Operate  radar 

0.12 

0.48 

Operate  conputer  hardware 

0.48 

0.86 

Cook 

0.22 

0.76 

Perform  medical  laboratory  procedures 

0.16 

0.82 

Conduct  land  surveys 

0.80 

1.16 

Provide  medical  or  dental  treatment 

0.16 

0.74 

Sketch  maps,  overlays,  or  range  cards 

1.38 

1.26 

Produce  technical  drawings 

0.28 

0.90 

Draw  maps  and  overlays 

0.40 

0.97 

Draw  illustrations 

0.52 

1.25 

Type 

0.74 

1.01 

Prepare  technical  forms  and  doconents 

2.48 

1.52 

Record,  file,  and  dispatch  information 

2.12 

1.61 

Receive,storc, (issue  si4^,  equip,  etc 

1.74 

1.48 

Use  hand  and  arm  signals 

2.56 

1.28 

Read  tech/field  marxials,  reg  etc 

4.10 

1.05 

Core  Technical  Gen.  Soldiering  Overall  Job 


N 

MEAN 

STO 

N 

HEAN 

STO 

N 

HEAN 

STO 

N 

50 

4.18 

1.06 

50 

3.78 

1.15 

50 

4.16 

0.93 

50 

50 

2.68 

1.68 

50 

3.84 

1.20 

50 

3.76 

1.25 

50 

50 

4.10 

1.16 

50 

2.64 

1.24 

50 

3.72 

1.16 

50 

50 

0.86 

1.46 

50 

1.04 

1.50 

50 

1.10 

1.62 

50 

50 

2.78 

1.30 

SO 

2.78 

1.30 

50 

3.90 

0.99 

50 

50 

0.90 

1.50 

49 

1.20 

1.62 

50 

1.14 

1.62 

49 

50 

1.59 

1.61 

49 

1.43 

1.51 

49 

1.78 

1.57 

49 

50 

3.60 

1.26 

so 

1.98 

1.29 

50 

3.16 

1.31 

50 

50 

1.50 

1.78 

SO 

0.94 

1.35 

50 

1.44 

1.75 

50 

50 

3.46 

1.30 

50 

1.84 

1.36 

50 

3.02 

1.36 

50 

50 

2.16 

1.80 

SO 

1.26 

1.48 

50 

1.86 

1.74 

50 

50 

1.44 

1.39 

50 

1.92 

1.50 

SO 

1.84 

1.45 

50 

50 

0.08 

0.57 

50 

0.08 

0.57 

SO 

0.08 

0.57 

50 

50 

0.32 

0,79 

50 

0.34 

0.85 

50 

0.32 

0.79 

50 

50 

0.38 

0.95 

50 

0.24 

0.74 

50 

0.24 

0.74 

50 

50 

4.16 

0.84 

50 

3.90 

0.95 

50 

4.20 

0.90 

50 

50 

1.80 

1.69 

50 

1.50 

1.64 

SO 

1.70 

1.72 

50 

50 

0.16 

0.79 

50 

0.16 

0.79 

50 

0.16 

0.79 

50 

50 

1.56 

1.50 

50 

1.16 

1.42 

50 

1.54 

1.49 

50 

50 

1.84 

1.43 

50 

1.68 

1.53 

50 

1.82 

1.48 

49 

50 

1.41 

1.48 

49 

1.47 

1.43 

49 

1.59 

1.50 

49 

50 

0.98 

1.39 

49 

0.55 

1.12 

49 

0.96 

1.43 

49 

50 

1.94 

1.42 

50 

0.88 

1.08 

50 

1.68 

1.38 

50 

50 

0.20 

0.73 

50 

0.20 

0.73 

50 

0.24 

0.74 

SO 

50 

0.34 

0.98 

50 

0.14 

0.61 

50 

0.32 

0.96 

50 

50 

0.26 

0.85 

50 

0.24 

0.85 

50 

0.26 

0.85 

50 

50 

0.16 

0.79 

50 

0.16 

0.79 

50 

0.16 

0.79 

50 

50 

3.14 

1.32 

50 

2.34 

1.35 

50 

2.90 

1.36 

50 

50 

0.08 

0.57 

50 

0.08 

0.57 

50 

0.08 

0.57 

50 

50 

0.38 

0.95 

SO 

0.54 

1.16 

50 

0.48 

1.03 

SO 

50 

0.78 

1.25 

50 

0.84 

1.25 

50 

0.88 

1.26 

SO 

50 

1.06 

1.43 

49 

1.04 

1.37 

49 

1.10 

1.42 

49 

50 

0.16 

0.68 

50 

0.16 

0.68 

50 

0.16 

0.68 

SO 

50 

0.64 

1.31 

SO 

0.60 

1.09 

50 

0.66 

1.22 

SO 

50 

0.28 

0.97 

50 

0.38 

1.07 

50 

0.38 

1.14 

SO 

50 

0.16 

0.82 

50 

0.16 

0.82 

50 

0.16 

0.82 

50 

50 

0.90 

1.47 

50 

1.12 

1.62 

SO 

1.10 

1.61 

SO 

50 

0.20 

0.90 

SO 

0.22 

0.91 

50 

0.22 

0.91 

SO 

50 

1.04 

1.38 

SO 

2.08 

1.82 

50 

1.78 

1.59 

SO 

50 

0.28 

0.97 

50 

0.30 

1.02 

50 

0.28 

0.97 

SO 

SO 

0.48 

1.16 

50 

0.68 

1.43 

50 

0.52 

1.22 

50 

50 

0.54 

1.27 

50 

0.54 

1.25 

50 

0.54 

1.27 

SO 

SO 

0.80 

1.21 

SO 

0.88 

1.30 

50 

0.94 

1.22 

50 

SO 

2.64 

1.68 

50 

2.02 

1.42 

50 

2.46 

1.54 

50 

50 

2.18 

1.60 

SO 

1.82 

1.52 

50 

2.16 

1.61 

50 

50 

2.02 

1.67 

50 

1.58 

1.53 

50 

2.00 

1.68 

50 

50 

2.66 

1.55 

50 

2.74 

1.26 

50 

2.82 

1.27 

50 

50 

4.36 

0.80 

SO 

3.76 

1.02 

50 

4.14 

0.90 

50 
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TASK  CATEGORY  (Contd) 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


MOS=63B 


Frequency 

Core  Technical 

Gen.  Soldier 

ing 

Overal 

1  Job 

Task  Categories 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Use  maps 

2.72 

1,29 

50 

2.78 

1.52 

50 

3.64 

1.12 

50 

3.54 

1.01 

50 

Send  and  receive  radio  messages 

2.24 

1.27 

50 

2.10 

1.69 

50 

3.18 

1.57 

50 

2.94 

1.45 

50 

Give  short  oral  reports 

2.06 

1.36 

50 

1.94 

1.52 

50 

2.54 

1.61 

50 

2.50 

1.46 

SO 

Receive  clients,  patients,  guests 

0.24 

0.82 

50 

0.28 

1.01 

50 

0.32 

1.04 

50 

0.32 

1.04 

50 

Give  directions  and  instructions 

2.34 

1.32 

50 

2.61 

1.27 

49 

2.71 

1.35 

49 

2.69 

1.33 

49 

Write  and  deliver  presentations 

0.34 

0.75 

50 

0,52 

1.16 

50 

0.60 

1.25 

50 

0.58 

1.26 

50 

Interview 

0.48 

0.95 

50 

0.66 

1.30 

50 

0.66 

1.26 

50 

0.68 

1.28 

50 

Provide  coixiseling 

0.68 

1.30 

50 

0.70 

1.31 

50 

0.78 

1.36 

50 

0.78 

1.39 

50 

Write  docunents  and  correspondence 

0.60 

1.20 

50 

0.58 

1.26 

50 

0.66 

1.30 

50 

0.66 

1.33 

50 

Decode  data 

0.96 

1.32 

50 

0.88 

1.36 

50 

1.44 

1.74 

50 

1.26 

1.58 

50 

Analyze  electronic  signals 

0.26 

0.72 

50 

0.28 

0.81 

50 

0.38 

0.99 

50 

0.30 

0.84 

50 

Analyze  weather  conditions 

0.42 

0.91 

50 

O.U 

1.03 

50 

0.52 

1.07 

50 

0.48 

1.05 

50 

Order  equipment  and  supplies 

1,94 

1.46 

50 

2.U 

1.69 

50 

1.76 

1.51 

50 

2.28 

1.63 

50 

Estimate  time  &  cost  of  maint  ops 

1,56 

1.54 

50 

1.76 

1.66 

50 

1.12 

1.49 

50 

1.58 

1.60 

50 

Plan  placement/use  of  tactical  equip 

1.22 

1,43 

50 

1.14 

1.53 

50 

1.56 

1.69 

50 

1.48 

1.62 

50 

Translate  foreign  languages 

0.24 

0.82 

50 

0.22 

0.74 

50 

0.32 

0.96 

SO 

0.30 

0.89 

50 

Analyze  intelligence  data 

0.50 

0.97 

50 

0.46 

0.97 

50 

0.64 

1.26 

50 

0.68 

1.35 

50 

Control  money 

0.56 

1.47 

50 

0.42 

1.21 

50 

0.58 

1,50 

SO 

0.56 

1.49 

50 

Determine  firing  data*indirect  weapons 

0.40 

1.03 

50 

0.42 

1.18 

50 

0.60 

1.40 

so 

0.56 

1.33 

SO 

Conpute  statistics/other  math 

0.38 

0.90 

50 

0.58 

1.28 

50 

0.50 

1.15 

50 

0.56 

1.20 

50 

Provide  progranming  t  DP  support 

0.26 

0.75 

50 

0.33 

1.11 

49 

0,29 

0,91 

49 

0.35 

1.11 

49 

Control  air  traffic 

0.04 

0.20 

50 

0.14 

0.70 

50 

0.14 

0.70 

50 

0.14 

0.70 

50 

Use  hand  grenades 

1.56 

1.30 

50 

1.39 

1.63 

49 

2.92 

1.80 

49 

2.61 

1.62 

49 

Protect  against  NBC  hazards 

2.86 

1.13 

50 

2.36 

1.78 

50 

4.20 

0.90 

50 

3.88 

1.00 

50 

Handle  demolitions  or  mines 

0.80 

0.93 

50 

0.84 

1.40 

50 

1.50 

1,71 

50 

1.42 

1.59 

50 

Engage  in  hand-to-hand  combat 

1.12 

1.04 

50 

1.04 

1.34 

50 

2.30 

1.72 

50 

1.94 

1.58 

50 

Fire  individual  weapons 

2.84 

1.17 

50 

2.34 

1.92 

50 

4.18 

0.94 

50 

4.02 

0.96 

50 

Control  individuals  and  crowds 

1.38 

1,38 

50 

1.14 

1.47 

50 

2.00 

1.93 

50 

1.78 

1.72 

50 

Customs  and  laws  of  war 

1.86 

1.26 

50 

1.54 

1.62 

50 

3.14 

1.57 

50 

2.84 

1.56 

50 

Navigate 

1.88 

1.21 

50 

1.76 

1.62 

50 

3.10 

1.59 

SO 

2.90 

1.57 

50 

Survive  in  the  field 

2.26 

1.38 

50 

1.88 

1.78 

50 

3.52 

1.61 

50 

3.34 

1.52 

50 

Move  and  react  in  the  field 

1.94 

1.24 

50 

1.78 

1.69 

50 

3.32 

1.43 

50 

3.16 

1.46 

50 

Load  t  unload  field  artillery/tank  gtrs 

0.22 

0.62 

50 

0.27 

0.97 

49 

0.29 

0.98 

49 

0.29 

0.98 

49 

Fire  heavy  direct  fire  weapons 

0.10 

0.42 

50 

0.22 

0.89 

50 

0.14 

0.70 

SO 

0.16 

0.79 

50 

Prepare  heavy  weapons  for  tactical  use 

0.14 

0.64 

50 

0.22 

0.89 

50 

0.20 

0.81 

SO 

0.22 

0.89 

50 

Place  t  camoufl  tactical  equip  and  mat 

1.86 

1.40 

50 

1.80 

1.70 

50 

2.72 

1.78 

50 

2.48 

1.69 

50 

Fire  indirect  fire  weapons 

0.08 

0.34 

49 

0.16 

0.72 

49 

0.16 

0.72 

49 

0.16 

0.72 

49 

Give  first  aid 

2.60 

1.11 

50 

2.50 

1.79 

50 

4.04 

0.92 

50 

4.00 

0.86 

SO 

Detect  and  identify  targets 

1.58 

1.30 

50 

1.46 

1.61 

50 

2.82 

1.67 

50 

2.62 

1.61 

50 

Plan  operations 

0.70 

1.33 

50 

0.70 

1.34 

50 

0.82 

1.53 

50 

0.88 

1.56 

50 

Direct/ lead  teams 

0.64 

0.96 

50 

0.88 

1.38 

50 

1.18 

1.67 

SO 

1.16 

1.65 

SO 

Moni tor/ inspect 

1.70 

1.57 

50 

1.86 

1.68 

50 

1.84 

1.72 

50 

1.90 

1.68 

SO 

Lead 

2.04 

1.64 

50 

2.22 

1.76 

49 

2.63 

1.93 

49 

2.49 

1.83 

49 

Act  as  a  modal 

2.64 

1.44 

50 

2.92 

1.67 

49 

3.45 

1.54 

49 

3.24 

1.52 

49 

Counsel 

1.52 

1.55 

50 

1.58 

1.72 

50 

2.00 

1.90 

SO 

1.84 

1.74 

SO 

CodiKricata 

2.32 

1.56 

50 

2.56 

1.55 

50 

2.96 

1.62 

50 

2.88 

1.59 

50 

Train 

1.56 

1.69 

50 

1.88 

2.00 

50 

2.00 

1.99 

SO 

1.98 

2.06 

SO 

Personnel  Adninstrstion 

1.00 

1.54 

50 

1.12 

1.72 

50 

1.26 

1,79 

50 

1.24 

1.78 

SO 
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TASK  CATEGCKY 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


MOS=71U 


Task  Categories 

Perf  operator  mint  chks  and  services 
Perf  operator  chks  t  services  on  weapons 
Troubleshoot  mechanical  systems 
Repair  weapons 
Repair  mechanical  syster.^ 

Troubleshoot  weapons 
Install  electronic  components 
Inspect  electrical  system 
Inspect  electronic  system 
Repair  electrical  system 
Repair  electronic  components 
Pack  and  load  raterials 
Prepare  parachutes 

Prepare  equip  and  supplies  for  air  drop 
Operate  power  excavating  equipmnt 
Operate  wheeled  vehicles 
Operate  track  vehicles 
Operate  boats 

Operate  lifting,loading,&grading  equip 
Paint 

Install  wire  and  cables 
Repair  plastic  and  fiberglass 
Repair  metal 

Assemble  steel  structures 
Install  pipe  assemblies 
Construct  wooden  bldgs  &  other  struct 
Construct  msonry  bldgs  t  structures 
Operate  gas  &  electric  powered  equip 
Select, layout. (clean  med/dental  equip 
Use  audiovisual  equipment 
Reproduce  printed  mterial 
Operate  electronic  equipment 
Operate  radar 
Operate  computer  hardiare 
Cook 

Perform  medical  laboratory  procedures 

Conduct  land  surveys 

Provide  medical  or  dental  treatment 

Sketch  maps,  overlays,  or  range  cards 

Produce  technical  drawings 

Draw  maps  and  overlays 

Draw  i llustrationa 

Type 

Prepare  technical  form  and  doevenents 
Record,  file,  and  dispatch  informtion 
Receive, store, (issue  sipp,  equip,  etc 
Use  hand  and  arm  tigrmls 
Read  tech/field  maruals,  reg  etc 


Frequency  Core  Technical 


MEAN 

STD 

N 

MEAN 

STD 

N 

1.56 

1.26 

52 

1.18 

1.32 

50 

1.77 

1.23 

52 

1.22 

1.19 

51 

0.31 

0.67 

52 

0.27 

0.66 

52 

0.25 

0.59 

52 

0.17 

0.51 

52 

0.21 

0.46 

52 

0.37 

0.86 

52 

0.21 

0.54 

52 

0.19 

0.53 

52 

0.85 

1.16 

52 

0.75 

1.14 

52 

0.40 

0.85 

52 

0.41 

0.96 

51 

0.27 

0.72 

52 

0.21 

0.64 

52 

0.27 

0.66 

52 

0.23 

0.61 

52 

0.29 

0.80 

52 

0.17 

0.51 

52 

1.00 

1.17 

52 

0.67 

1.02 

52 

0.08 

0.33 

52 

0.06 

0.31 

52 

0.12 

0.32 

52 

0.08 

0.27 

52 

0.02 

0.14 

52 

0.02 

0.14 

52 

1.79 

1.14 

52 

1.31 

1.39 

51 

0.15 

0.46 

52 

0.06 

0.24 

51 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.08 

0.34 

51 

0.08 

0.44 

51 

0.87 

0.97 

52 

0.23 

0.51 

52 

0.46 

0.83 

52 

0.43 

0.88 

51 

0.08 

0.27 

52 

0.02 

0.14 

51 

0.12 

0.32 

52 

0.04 

0.20 

51 

0.12 

0.43 

52 

0.10 

0.50 

51 

0.04 

0.19 

52 

0.02 

0.14 

51 

0.08 

0.27 

52 

0.04 

0.20 

51 

0.02 

0.14 

52 

0.00 

0.00 

51 

0.92 

1.03 

52 

0.78 

1.14 

51 

0.04 

0.28 

52 

0.00 

0.00 

51 

0.73 

1.12 

52 

0.55 

1.03 

51 

3.04 

1,57 

52 

2.69 

1.77 

51 

1,92 

1.77 

52 

1.94 

1.93 

51 

0.02 

0.14 

52 

0.00 

0.00 

51 

3.08 

1.57 

52 

2.88 

1.66 

51 

0.10 

0.36 

52 

0.04 

0.19 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.29 

0.67 

52 

0.10 

0.36 

51 

0.08 

0.33 

52 

0.08 

0.39 

52 

0.83 

0.98 

52 

0.43 

0.83 

51 

0.13 

0.53 

52 

0.06 

0.31 

51 

0.19 

0.53 

52 

0,13 

0.53 

52 

0.17 

0.51 

52 

0.17 

0.55 

52 

4,77 

0.61 

52 

4.75 

0.56 

51 

4.15 

1.13 

52 

4.02 

1.08 

52 

4.46 

0.83 

52 

4.25 

0.84 

52 

2.83 

1,52 

52 

2.56 

1.41 

52 

1.06 

0.92 

52 

0.63 

0.87 

51 

3.52 

1.35 

52 

3.63 

1.34 

52 

Gen.  Soldiering 

Overal 

1  Job 

MEAN 

STO 

N 

MEAN 

STO 

N 

2,37 

1.70 

51 

2.12 

1.60 

51 

3,47 

1.55 

51 

2.96 

1.48 

51 

0.54 

1.15 

52 

0.46 

1.00 

52 

0.52 

1.20 

52 

0.42 

1.04 

52 

0,37 

0.86 

52 

0.35 

0.81 

52 

0.48 

1.20 

52 

0.44 

1.11 

52 

1.10 

1.42 

52 

1.08 

1.41 

52 

0.41 

1.02 

51 

0.53 

1.16 

51 

0.33 

0.92 

52 

0.35 

0.88 

52 

0.25 

0.71 

52 

0.25 

0.65 

52 

0.21 

0.70 

52 

0.25 

0.71 

52 

1.40 

1.52 

52 

1.27 

1.42 

52 

0.17 

0.81 

52 

0,13 

0.71 

52 

0.19 

0.60 

52 

0.13 

0.40 

52 

0.06 

0.42 

52 

0.06 

0.42 

52 

2.44 

1.50 

52 

2.27 

1.50 

52 

0.19 

0.66 

52 

0,17 

0.62 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.10 

0.57 

51 

0.10 

0.57 

51 

0.60 

0.75 

52 

0.54 

0.83 

52 

0.71 

1.29 

52 

0.63 

1.09 

52 

0.08 

0.33 

52 

0.08 

0.33 

52 

0.08 

0.27 

51 

0.08 

0.27 

51 

0.19 

0.72 

52 

0.17 

0.65 

52 

0.02 

0.14 

51 

0.02 

0.14 

51 

0.12 

0.43 

52 

0.12 

0.43 

52 

0.04 

0.28 

52 

0.04 

0.28 

52 

1.31 

1.44 

52 

1.12 

1.29 

52 

0.10 

0.69 

52 

0.10 

0.69 

52 

0.62 

1.07 

52 

0.79 

1.18 

52 

1.33 

1.46 

52 

2.63 

1.67 

52 

1.48 

1.64 

52 

2.10 

1.81 

52 

0.08 

0.55 

52 

0.08 

0.55 

52 

1.59 

1.63 

51 

2.88 

1.68 

52 

0.06 

0.24 

52 

0.04 

0.19 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0,55 

1.25 

51 

0.39 

0.98 

51 

0.23 

0.94 

52 

0.19 

0.79 

52 

1.44 

1.66 

52 

1.17 

1.40 

52 

0.21 

0.78 

52 

0.13 

0.53 

52 

0.25 

0.79 

52 

0.19 

0.63 

52 

0.25 

0.76 

52 

0.17 

0.51 

52 

1.94 

1.67 

52 

4.19 

1.14 

52 

2.00 

1.57 

52 

3.69 

1.12 

51 

2.15 

1.61 

52 

3.92 

1.01 

52 

2.08 

1.48 

52 

2.71 

1.26 

52 

2.13 

1.73 

52 

1.63 

1.39 

52 

3.48 

1.26 

52 

3.65 

1.25 

52 
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TASK  CATEGORY  (Contd) 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


Frequency 

Core  Technical 

Gen.  Soldiering 

Overall  Job 

Task  Categories 

HEAN 

STD 

N 

MEAN 

STD 

N 

HEAN 

STD 

N 

MEAN 

STD 

N 

Use  maps 

1.87 

1.16 

52 

1.37 

1.40 

51 

3.52 

1.38 

52 

2.88 

1.46 

52 

Send  and  receive  radio  messages 

1.22 

1.01 

51 

0.84 

1.13 

50 

2.41 

1.80 

51 

1.92 

1.55 

51 

Give  short  oral  reports 

1.37 

1.09 

52 

1.37 

1.28 

51 

2.38 

1.60 

52 

2.19 

1.53 

52 

Receive  clients,  patients,  guests 

1.54 

1.72 

52 

1.25 

1.53 

52 

0.79 

1.43 

52 

1.35 

1.57 

52 

Give  directions  and  instructions 

3.00 

1.31 

52 

2.82 

1.29 

51 

2.77 

1.48 

52 

3.19 

1.17 

52 

Write  and  deliver  presentations 

1.08 

1.33 

52 

1.08 

1.41 

51 

0.86 

1.33 

51 

1.18 

1.38 

51 

Interview 

1.31 

1.54 

52 

1.35 

1.57 

52 

1.31 

1.52 

52 

1.48 

1.54 

52 

Provide  counseling 

0.88 

1.35 

52 

0.86 

1.40 

51 

1.04 

1.53 

52 

1.17 

1.61 

52 

Write  docunents  and  correspondence 

2.75 

1.67 

52 

2.90 

1.64 

51 

1.73 

1.57 

52 

2.84 

1.65 

51 

Decode  data 

0.58 

0.72 

52 

0.48 

0.96 

52 

1.23 

1.54 

52 

1.13 

1.43 

52 

Analyze  electronic  signals 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

Analyze  weather  conditions 

0.06 

0.31 

52 

0.02 

0.14 

52 

0.08 

0.44 

52 

0.06 

0.42 

52 

Order  equipment  tnd  supplies 

1.88 

1.34 

52 

1.51 

1.24 

51 

1.35 

1.36 

52 

1.71 

1.32 

52 

Estimate  time  t  cost  of  maint  ops 

0.27 

0.60 

52 

0.14 

0.53 

51 

0.38 

1.03 

52 

0.35 

0.95 

52 

Plan  placement/use  of  tactical  equip 

0.27 

0.53 

52 

0.16 

0.46 

51 

0.60 

1.27 

52 

0.44 

0.98 

52 

Translate  foreign  languages 

0.19 

0.53 

52 

0.16 

0.73 

51 

0.21 

0.64 

52 

0.25 

0.68 

52 

Analyze  intelligence  data 

0.06 

0.24 

52 

0.02 

0.14 

52 

0.10 

0.41 

52 

0.06 

0.31 

52 

Control  money 

0.50 

0.85 

52 

0.55 

1.19 

51 

0.50 

1.00 

52 

0.63 

1.24 

52 

Determine  firing  data*irKlirect  weapons 

0.10 

0.41 

52 

0.08 

0.39 

52 

0.12 

0.51 

52 

0.08 

0.39 

52 

Conpute  statistics/other  math 

0.71 

1.07 

52 

0.83 

1.28 

52 

0.50 

0.96 

52 

0.85 

1.26 

52 

Provide  programing  t  DP  sipport 

1.15 

1.70 

52 

1.18 

1.58 

51 

0.69 

1.18 

52 

1.12 

1.44 

52 

Control  air  traffic 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

Use  hard  greruKles 

1.13 

0.91 

52 

0.64 

1.01 

50 

2.65 

1.67 

52 

2.25 

1.60 

52 

Protect  against  NBC  hazards 

2.04 

1.07 

52 

1.24 

1.39 

49 

4.02 

1.13 

52 

3.37 

1.36 

52 

Handle  demolitions  or  mir«s 

0.33 

0.68 

52 

0.12 

0.38 

52 

0.69 

1.42 

52 

0.52 

1.18 

52 

Engage  in  hand-to-hand  combat 

0.69 

1.02 

52 

0.29 

0.67 

51 

1.52 

1.89 

52 

1.25 

1.69 

52 

Fire  individual  weapons 

1.98 

1.02 

52 

1.18 

1.40 

SO 

4.31 

0.90 

52 

3.56 

1.19 

52 

Control  individuals  and  crowds 

0.98 

1.04 

52 

0.45 

0.90 

51 

1.96 

1.84 

52 

1.65 

1.64 

52 

Customs  and  laws  of  war 

1.38 

1.07 

52 

0,90 

1.30 

50 

3.10 

1.50 

52 

2.54 

1.46 

52 

Navigate 

1.40 

1.05 

52 

0.82 

1.13 

49 

3.25 

1.71 

51 

2.71 

1.50 

51 

Survive  in  the  field 

1.77 

1.02 

52 

1.24 

1.51 

50 

3.98 

1.20 

52 

3.35 

1.33 

52 

Hove  and  react  in  the  field 

1.60 

1.14 

52 

0.98 

1.24 

SO 

3.56 

1.66 

52 

3.06 

1.55 

52 

Load  t  mload  field  artillery/tank  gum 

0.02 

0.14 

52 

0.00 

0.00 

52 

0.06 

0.42 

52 

0.06 

0.42 

52 

Fire  heavy  direct  fire  weapons 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

Prepare  heavy  weapons  for  tactical  use 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

Place  t  camoufl  tactical  equip  and  mat 

0.92 

1.08 

52 

0.57 

1.19 

51 

1.84 

1.94 

51 

1.51 

1.74 

51 

Fire  indirect  fire  weapons 

0.04 

0.28 

52 

0.04 

0.28 

52 

0.06 

0.42 

52 

0.06 

0.42 

52 

Give  first  aid 

1.88 

1.00 

52 

1.22 

1.39 

50 

4.17 

0.81 

52 

3.54 

1.09 

52 

Detect  and  identify  targets 

1.06 

0.98 

52 

0.60 

0.95 

50 

2.37 

1.84 

52 

1.94 

1.60 

52 

Plan  operations 

0.44 

1.06 

52 

0.41 

1.08 

51 

0.63 

1.31 

52 

0.60 

1.29 

52 

Oirect/lead  temas 

0.46 

0.73 

52 

0.35 

0.71 

52 

1.04 

1.55 

52 

0.90 

1.45 

52 

Monitor/ inspect 

1.38 

1.66 

52 

1.39 

1.78 

51 

1.85 

1.96 

52 

1.75 

1.86 

52 

Lead 

1.94 

1.81 

52 

1.94 

1.89 

50 

2.47 

2.03 

51 

2.37 

2.05 

52 

Act  as  a  model 

2.52 

1.85 

52 

2.44 

1.88 

50 

2.87 

1.87 

52 

2.98 

1.82 

52 

Couisel 

1.85 

1.73 

52 

1.75 

1.81 

51 

2.17 

1.97 

52 

2.15 

1.99 

52 

Connsricate 

2.50 

1,93 

52 

2.37 

1.94 

51 

2.63 

1.99 

52 

2.65 

1.96 

52 

Train 

1.62 

1.72 

52 

1.)3 

1.95 

51 

2.10 

2.01 

52 

2.12 

2.01 

52 

Personnel  Actainstration 

2.38 

1.90 

52 

2.41 

1.99 

51 

1.98 

1.95 

52 

2.38 

1.89 

52 
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TASK  CATtGORY 

Frequency  and  Iirportance  Ratings  Descriptive  Statistics 


Frequency 

Core  Technical 

Gen.  Soldiering 

Overall  Job 

Task  Categories 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Perf  operator  mint  chks  and  services 

3.22 

1.65 

190 

3.24 

1.80 

188 

3.30 

1.56 

188 

3.47 

1.57 

187 

Perf  operator  chks  t  services  on  weapons 

3.U 

1.50 

190 

3.13 

1.84 

189 

3.90 

1.30 

189 

3.85 

1.31 

189 

Troubleshoot  mechanical  systems 

1.84 

1.87 

190 

2.04 

2.00 

190 

1.65 

1.65 

190 

2.00 

1.92 

190 

Repair  weapons 

0.94 

1.27 

189 

1.13 

1.59 

189 

1.20 

1.58 

189 

1.26 

1.66 

189 

Repair  mechanical  systems 

1.90 

1.90 

190 

1.66 

1.68 

190 

1.66 

1.68 

190 

2.02 

1.92 

190 

Troubleshoot  weapons 

1.34 

1.58 

190 

1.60 

1.89 

189 

1.61 

1.79 

190 

1.65 

1.81 

189 

Install  electronic  components 

1.67 

1.62 

190 

1.76 

1.76 

189 

1.76 

1.70 

189 

1.85 

1.72 

189 

Inspect  electrical  systems 

1.23 

1.68 

190 

1.34 

1.82 

188 

0.88 

1.33 

188 

1.22 

1.67 

188 

Inspect  electronic  systems 

0.69 

1.29 

190 

0.74 

1.38 

189 

0.58 

1.14 

189 

0.73 

1.36 

189 

Repair  electrical  systems 

1.06 

1.51 

190 

1.20 

1.71 

189 

0.78 

1.24 

189 

1.12 

1.59 

189 

Repair  electronic  components 

0.73 

1.28 

190 

0.80 

1.44 

190 

0.58 

1.16 

190 

0.75 

1.34 

190 

Pack  and  load  mterials 

1.66 

1.47 

190 

1.53 

1.51 

189 

1.78 

1.53 

189 

1.74 

1.51 

189 

Prepare  parachutes 

0.20 

0.71 

189 

0.20 

0.77 

189 

0.24 

0.85 

189 

0.23 

0.84 

189 

Prepare  equip  and  supplies  for  air  drop 

0.36 

0.78 

190 

0.43 

1.01 

190 

0.47 

1.07 

190 

0.45 

1.05 

190 

Operate  power  excavating  equipment 

0.15 

0.54 

189 

0.18 

0.63 

189 

0.14 

0.58 

189 

0.15 

0.59 

189 

Operate  wheeled  vehicles 

2.69 

1.51 

190 

2.70 

1.65 

188 

2.89 

1.49 

189 

2.99 

1.57 

189 

Operate  track  vehicles 

1.68 

1.90 

190 

1.78 

1.98 

189 

1.57 

1.81 

190 

1.73 

1.92 

190 

Operate  boats 

0.23 

0.68 

190 

0.32 

0.94 

190 

0.27 

0.85 

190 

0.28 

0.83 

190 

Operate  lifting, loading, (grading  equip 

0.47 

1.02 

189 

0.49 

1.10 

188 

0.38 

0.97 

188 

0.48 

1.09 

188 

Paint 

1.46 

1.24 

190 

1.04 

1.27 

188 

1.19 

1.27 

188 

1.19 

1.27 

187 

Install  wire  and  cables 

1.15 

1.32 

190 

1.34 

1.57 

186 

1.34 

1.53 

186 

1.35 

1.53 

186 

Repair  plastic  and  fiberglass 

0.29 

0.73 

190 

0.34 

0.90 

188 

0.25 

0.74 

189 

0.33 

0.88 

189 

Repair  metal 

0.55 

0.95 

190 

0.60 

1.15 

189 

0.35 

0.79 

189 

0.58 

1.09 

189 

Assenble  steel  structures 

0.18 

0.59 

190 

0.20 

0.71 

189 

0.23 

0.75 

190 

0.24 

0.75 

190 

Install  pipe  assemblies 

0.12 

0.48 

190 

0.16 

0.69 

189 

0.11 

0.52 

189 

0.16 

0.68 

189 

Construct  wooden  bldgs  (  other  struct 

0.20 

0.60 

190 

0.19 

0.68 

188 

0.23 

0.72 

189 

0.23 

0.73 

189 

Construct  msonry  bldgs  (  structures 

0.07 

0.39 

190 

0.09 

0.53 

189 

0.10 

0.55 

190 

0.11 

0.59 

190 

Operate  gas  (  electric  poMred  equip 

1.28 

1.40 

190 

1.32 

1.57 

189 

1.26 

1.42 

190 

1.36 

1.54 

190 

Select, layout, (clean  med/dental  equip 

0.06 

0.37 

190 

0.05 

0.38 

189 

0.08 

0.55 

190 

0.08 

0.55 

190 

Use  audiovisual  equipment 

0.59 

0.99 

190 

0.55 

1.08 

188 

0.65 

1.13 

189 

0.67 

1.14 

189 

Reproduce  printed  mterial 

1.44 

1.65 

190 

1.18 

1.59 

188 

0.93 

1.30 

189 

1.28 

1.56 

189 

Operate  electronic  equipment 

1.36 

1.60 

190 

1.46 

1.75 

188 

1.26 

1.57 

189 

1.49 

1.74 

188 

Operate  radar 

0.09 

0.51 

190 

0.09 

0.49 

189 

0.12 

0.58 

190 

0.12 

0.61 

190 

Operate  computer  hardware 

1.15 

1.61 

190 

1.11 

1.65 

189 

0.80 

1.32 

189 

1.19 

1.68 

190 

Cook 

0.27 

0.88 

189 

0.25 

0.88 

189 

0.29 

0.95 

189 

0.29 

0.95 

189 

Perform  medical  laboratory  procedures 

0.13 

0.67 

190 

0.13 

0.69 

190 

0.12 

0.66 

190 

0.13 

0.69 

190 

Conduct  land  surveys 

0.72 

1.34 

189 

0.74 

1.48 

187 

0.85 

1.51 

187 

0.81 

1.47 

187 

Provide  medical  or  dental  treatment 

0.19 

0.71 

189 

0.22 

0.83 

188 

0.28 

0.96 

188 

0.25 

0.91 

188 

Sketch  maps,  overlays,  or  range  cards 

2.13 

1.71 

190 

2.06 

1.97 

188 

2.37 

1.81 

189 

2.37 

1.81 

189 

Produce  technical  drawings 

0.18 

0.67 

190 

0.17 

0.70 

189 

0.22 

0.82 

190 

0.18 

0.72 

190 

Draw  maps  and  overlays 

0.57 

1.16 

190 

0.68 

1.40 

190 

0.69 

1.35 

190 

0.64 

1.29 

190 

Draw  illustrations 

O.U 

1.05 

190 

0.49 

1.17 

189 

0.49 

1.14 

189 

0.44 

1.07 

189 

Type 

1.83 

2.05 

190 

1.75 

2.07 

189 

1.08 

1.48 

189 

1.64 

1.92 

190 

Prepare  technical  forms  and  doosnents 

2.27 

1.88 

190 

2.26 

1.94 

189 

1.56 

1.58 

189 

2.10 

1.80 

188 

Record,  file,  and  dispatch  information 

2.20 

1.95 

190 

2.12 

1.91 

189 

1.49 

1.57 

189 

2.06 

1.84 

189 

Receive, store,({ssue  si^,  equip,  etc 

1.65 

1.64 

190 

1.66 

1.63 

189 

1.42 

1.54 

189 

1.74 

1.64 

189 

Use  hand  and  arm  signals 

2.86 

1.65 

190 

2.86 

1.92 

188 

3.07 

1.58 

189 

3.06 

1.60 

189 

Read  tech/field  manuals,  reg  etc 

3.74 

1.18 

190 

3.93 

1.15 

189 

3.70 

1.17 

189 

3.87 

1.12 

189 
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TASK  CATEGORY  (Contd) 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


HOS^ALL 


Frequency 


Core  Technical  Gen.  Soldiering  Overall  Job 


Task  Categories 

MEAN 

STD 

M 

HEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Use  maps 

3.09 

1.49 

190 

3.17 

1.77 

188 

3.76 

1.21 

189 

3.62 

1.28 

189 

Send  and  receive  radio  messages 

2.59 

1.52 

189 

2.74 

1.90 

187 

3.25 

1.56 

188 

3.11 

1.54 

188 

Give  short  oral  reports 

2.66 

1.59 

190 

2.84 

1.80 

188 

3.05 

1.61 

189 

3.09 

1.55 

189 

Receive  clients,  patients,  guests 

0.63 

1.29 

190 

0.54 

1.19 

189 

0.44 

1.10 

189 

0.60 

1.24 

189 

Give  directions  and  instructions 

2.72 

1.45 

189 

2.88 

1.47 

186 

2.80 

1.54 

187 

2.97 

1.48 

187 

Write  and  deliver  presentations 

0.65 

1.15 

190 

0.73 

1.35 

188 

0.68 

1.26 

188 

0,77 

1.35 

188 

Interview 

0.73 

1.28 

190 

0.83 

1.41 

190 

0.81 

1.39 

190 

0.87 

1.41 

190 

Provide  cocrseling 

0.97 

1.48 

190 

1.01 

1.58 

188 

1.07 

1.58 

189 

1.14 

1.68 

189 

Write  docunents  and  correspondence 

1.22 

1.65 

190 

1.31 

1.77 

189 

1.01 

1.54 

190 

1,27 

1.73 

189 

Decode  data 

1.41 

1.43 

190 

1.65 

1.79 

188 

1.88 

1.73 

188 

1.80 

1.70 

187 

Analyze  electronic  signals 

0.22 

0.72 

190 

0.31 

0.95 

190 

0.29 

0.88 

190 

0.28 

0.87 

190 

Analyze  weather  conditions 

0.54 

1.13 

190 

0.65 

1.42 

189 

0.57 

1.22 

189 

0.59 

1.30 

189 

Order  equipment  and  supplies 

1.50 

1.46 

190 

1.65 

1.67 

187 

1.37 

1.53 

188 

1.60 

1.60 

188 

Estimate  time  k  cost  of  maint  ops 

0.74 

1.37 

190 

0.72 

1.40 

187 

0.56 

1.24 

187 

0,71 

1.35 

187 

Plan  placement/use  of  tactical  equip 

1.22 

1.61 

190 

1.32 

1.82 

187 

1.39 

1.73 

188 

1.43 

1.75 

188 

Translate  foreign  languages 

0.23 

0.77 

190 

0.20 

0.75 

188 

0.26 

0.88 

189 

0.25 

0.78 

189 

Analyze  intelligence  data 

0.67 

1.24 

190 

0.77 

1.42 

188 

0.76 

1.35 

188 

0.78 

1.41 

188 

Control  money 

0.65 

1.32 

190 

0.63 

1.39 

188 

0.67 

1.41 

189 

0,72 

1.49 

189 

Determine  firing  data- indirect  weapons 

0.62 

1.26 

190 

0.70 

1.46 

188 

0.67 

1.38 

188 

0.69 

1.42 

188 

Conpute  statistics/other  math 

0.52 

1.02 

190 

0.62 

1.25 

188 

0.51 

1.09 

188 

0.62 

1.20 

188 

Provide  progranning  &  DP  sipport 

0.48 

1.13 

190 

0.53 

1.24 

186 

0.37 

0.96 

187 

0.48 

1,10 

187 

Control  air  traffic 

0.08 

0.40 

190 

0.15 

0.69 

190 

0.12 

0.56 

190 

0.13 

0.63 

190 

Use  hand  grenades 

2.28 

1.55 

190 

2.50 

2.04 

185 

3.25 

1.62 

187 

3,18 

1.60 

187 

Protect  against  NBC  hazards 

3.07 

1.32 

190 

3.12 

1.91 

185 

4.22 

0.96 

188 

3.98 

1.14 

188 

Handle  demolitions  or  mines 

1.42 

1.52 

190 

1.79 

2.01 

189 

1.89 

1.85 

189 

1.93 

1.89 

189 

Engage  in  hand-to-hand  combat 

1.55 

1.45 

190 

1.92 

1.94 

188 

2.44 

1.85 

189 

2.34 

1.81 

189 

Fire  individual  weapons 

3.23 

1.42 

190 

3.11 

1.98 

187 

4.33 

0.94 

189 

4.12 

1.03 

189 

Control  individuals  and  crowds 

2.01 

1.59 

190 

1.97 

1.87 

188 

2.50 

1.84 

189 

2.48 

1.79 

189 

Customs  and  laws  of  war 

2.27 

1.48 

190 

2.29 

1.87 

185 

3.28 

1.46 

188 

3.04 

1.50 

188 

Navigata 

2.58 

1.58 

190 

2.70 

1.98 

186 

3.51 

1.53 

188 

3.38 

1.51 

188 

Survive  in  the  field 

2.95 

1.62 

190 

2.85 

2.00 

187 

3.86 

1.41 

188 

3.72 

1.40 

189 

Hove  and  react  in  the  field 

2.93 

1.67 

190 

2.88 

2.02 

187 

3.72 

1.44 

189 

3.69 

1.42 

189 

Load  t  unload  field  artillery/tank  gisis 

0.35 

1.03 

189 

0.40 

1.22 

188 

0.37 

1.09 

188 

0.40 

1.18 

188 

Fire  heavy  direct  fire  weapona 

0.76 

1.46 

190 

1.00 

1.79 

190 

0.80 

1.55 

190 

0.91 

1.69 

190 

Prepare  heavy  weapons  for  tactical  use 

0.32 

1.02 

190 

0.38 

1.20 

190 

0.33 

1.06 

190 

0.35 

1.12 

190 

Place  t  camoufl  tactical  equip  and  mat 

2.12 

1.72 

190 

2.26 

2.02 

189 

2.66 

1.85 

189 

2.63 

1.88 

189 

Fire  indirect  fire  weapons 

0.41 

1.13 

189 

0.44 

1.23 

188 

0.46 

1.26 

188 

0.45 

1.25 

188 

Give  first  aid 

2.78 

1.28 

190 

3.01 

1.88 

186 

4.19 

0.90 

188 

4.02 

0.96 

188 

Detect  and  identify  targets 

2.39 

1.65 

190 

2.54 

2.02 

187 

3.17 

1.65 

189 

3.09 

1.66 

189 

Plan  operations 

1.08 

1.57 

190 

1.32 

1.87 

188 

1.34 

1.84 

189 

1.37 

1.87 

189 

Direct/lead  teams 

1.27 

1.59 

190 

1.57 

1.94 

189 

1.65 

1.85 

189 

1.67 

1.90 

189 

Honi tor/ inspect 

1.92 

1.79 

190 

2.12 

1.93 

188 

2.18 

1.92 

189 

2.22 

1.92 

189 

Lead 

2.25 

1.77 

190 

2.51 

1.95 

185 

2.68 

1.97 

186 

2.70 

1.97 

187 

Act  as  a  model 

2.67 

1.72 

190 

2.90 

1.87 

185 

3.12 

1.81 

187 

3.14 

1.79 

187 

Courmel 

1.88 

1.77 

190 

2.06 

1.95 

188 

2.26 

1.98 

189 

2.24 

1.99 

189 

CoRininlcate 

2.57 

1.85 

190 

2.78 

1.92 

188 

2.85 

1.91 

189 

2.90 

1.90 

188 

Train 

1.79 

1.83 

190 

2.10 

2.09 

188 

2.18 

2.04 

189 

2.22 

2.09 

189 

Personnel  Adainstration 

1.65 

1.89 

190 

1.72 

1.99 

188 

1.61 

1.92 

189 

1.74 

1.96 

189 
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JOB  ACTIVITY 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


H0S=11B 

Job  Activities 

Work  in  a  team 

Lead  a  team 

Support/advise  peers 

Support/advise  subordinates 

Coach  peers 

Coach  subordinates 

Hake  oral  reports  (to  individuals) 

Hake  oral  reports  (to  groups) 

Relay  oral  instructions 
Interview 

Record  information 

Write  brief  messages 

Write  longer  reports 

Honi tor/ interpret  verbal  messages 

Recall  verbal  information 

Honi tor/ interpret  rajnerical  information 

Recall  nunerical  information 

Honi tor/ interpret  figural  information 

Recall  figural  information 

Follow  oral  directions 

Follow  written  directions 

Judge  size  and  distance 

Judge  location 

Judge  paths  of  movir>g  objects 

Solve  electrical  system  problems 

Solve  mechanical  system  problems 

Solve  logistical  problems 

Solve  tactical  maneuver  problems 

Solve  adninistrative  problems 

Solve  leadership  problems 

Solve  medical  probleam 

Solve  conmLnication  problems 

Operate  precision  hand'hcld  equipment 

Operate  hand-held  tools 

Operate  hand-held  power  equipment 

Operate  larger  power  equipment 

Operate  full  keyboard 

Operate  numeric  keyboard 

Adjust  control  device  using  orw  liido 

Adj  control  device  using  isult  linbs 

Drive  tracked  vehicle 

Drive  heavy  tiieeled  vehicle 

Drive  light  wheeled  vehicle 

Aim:stationary  target 

Aim:moving  target 

Walk  long  distances 

Run  short  distances 

Push,  pull,  lift  heavy  waists 

Throw  objects 

Sort,  fold,  feed  by  hand 

Hake  coordinated  movements 

Work  long  hours 

Work  laider  adverse  conditions 


Frequency 

Core  Technical 

HEAN 

STD 

M 

HEAN 

STD 

N 

4.18 

0.90 

88 

4.37 

0.98 

87 

2.28 

1.47 

88 

3.07 

1.66 

87 

2.94 

1.46 

88 

3.^ 

1.51 

87 

2.60 

1.58 

88 

2.94 

1.67 

87 

2.7U 

1.U 

88 

2.97 

1.56 

87 

2.67 

1.57 

86 

2.99 

1.64 

84 

2.70 

1.54 

88 

2.76 

1.69 

87 

1.51 

1.41 

88 

1.74 

1.74 

87 

3.07 

1.51 

88 

3.26 

1.57 

87 

1.38 

1.51 

88 

1.54 

1.70 

87 

2.49 

1.41 

88 

2.67 

1.51 

87 

1.95 

1.55 

88 

2.06 

1.74 

87 

0.64 

1.01 

87 

0.88 

1.40 

86 

2.13 

1.77 

88 

2.38 

1.78 

86 

2.70 

1.63 

88 

2.84 

1.71 

86 

1.44 

1.49 

88 

1.57 

1.55 

86 

1.74 

1.58 

88 

1.79 

1.65 

86 

1.64 

1.41 

88 

2.00 

1.74 

86 

1.68 

1.50 

88 

1.88 

1.75 

86 

3.86 

1.45  . 

88 

3.84 

1.49 

86 

3.41 

1.31 

88 

3.65 

1.36 

86 

3.00 

1.37 

88 

3.55 

1.42 

86 

3.00 

1.41 

88 

3.71 

1.56 

86 

2.20 

1.57 

88 

2.65 

1.83 

86 

0.97 

1.35 

88 

1.08 

1.53 

87 

1.51 

1.49 

88 

1.79 

1.77 

86 

0.80 

1.22 

88 

0.94 

1.39 

86 

1.75 

1.64 

88 

2.27 

1.92 

86 

1.00 

1.29 

88 

1.23 

1.61 

86 

1.65 

1.63 

88 

2.01 

1.86 

86 

0.85 

1.22 

88 

0.98 

1.46 

86 

1.58 

1.48 

88 

1.87 

1.68 

87 

0.69 

1.43 

88 

0.72 

1.43 

88 

2.38 

1.56 

87 

2.30 

1.60 

86 

0.86 

1.25 

87 

0.88 

1.22 

86 

0.59 

1.27 

86 

0.60 

1.28 

86 

0.53 

0.91 

87 

0.46 

0.96 

87 

0.32 

0.74 

87 

0.34 

0.78 

87 

2.01 

1.76 

87 

2.09 

1.87 

87 

1.89 

1.82 

87 

1.95 

1.85 

86 

2.54 

1.86 

87 

2.68 

1.89 

87 

1.52 

1.61 

87 

1.67 

1.75 

87 

2.20 

1.46 

87 

2.46 

1.59 

87 

4.10 

1.00 

87 

4.56 

0.85 

86 

3.66 

1.35 

87 

4.32 

1.12 

87 

4.10 

1.12 

87 

4.17 

1.21 

86 

4.22 

1.05 

87 

4.24 

1.03 

86 

3.48 

1.33 

87 

3.51 

1.47 

86 

3.23 

1.36 

87 

3.45 

1.46 

86 

1.48 

1.72 

87 

1.47 

1.82 

87 

2.69 

1.73 

87 

2.92 

1.75 

86 

4.21 

0.86 

87 

3.74 

1.36 

86 

4.07 

1.11 

86 

3.95 

1.34 

85 

Gen. 

Soldiering 

Overall  Job 

HEAN 

STD 

N 

HEAN 

STD 

N 

4.22 

1.11 

87 

4.46 

0.83 

87 

2.89 

1.60 

87 

3.14 

1.63 

87 

3.20 

1.43 

87 

3.32 

1.43 

87 

2.98 

1.67 

87 

3.03 

1.71 

87 

3.01 

1.56 

87 

3.08 

1.54 

87 

3.01 

1.65 

84 

3.15 

1.71 

84 

2.82 

1.65 

87 

2.84 

1.61 

87 

1.82 

1.73 

87 

1.91 

1.74 

87 

3.16 

1.55 

87 

3.31 

1.50 

87 

1.52 

1.67 

87 

1.54 

1.67 

87 

2.68 

1.57 

87 

2.80 

1.58 

87 

2.17 

1.78 

87 

2.22 

1.79 

87 

0.79 

1.29 

86 

0.84 

1.33 

86 

2.26 

1.75 

85 

2.32 

1.75 

85 

2.69 

1.59 

85 

2.78 

1.61 

85 

1.54 

1.52 

85 

1.59 

1.60 

85 

1.85 

1.63 

85 

1.86 

1.66 

85 

1.84 

1.62 

85 

1.82 

1.60 

85 

1.71 

1.58 

85 

1.88 

1.73 

85 

3.84 

1.44 

85 

4.00 

1.39 

85 

3.72 

1.35 

85 

3.84 

1.38 

85 

2.94 

1.34 

86 

3.26 

1.29 

86 

3.08 

1.54 

86 

3.35 

1.49 

86 

2.19 

1.61 

86 

2.38 

1.65 

86 

1.01 

1.42 

87 

1.01 

1.40 

87 

1.57 

1.59 

86 

1.69 

1.66 

86 

0.91 

1.34 

86 

0.93 

1.38 

86 

1.87 

1.74 

86 

2.15 

1.89 

86 

1.28 

1.67 

86 

1.36 

1.71 

86 

1.88 

1.81 

86 

2.07 

1.92 

86 

0.93 

1.43 

86 

1.02 

1.56 

86 

1.84 

1.66 

87 

1.91 

1.73 

87 

0.66 

1.35 

88 

0.72 

1.43 

88 

2.16 

1.47 

86 

2.29 

1.54 

86 

0.93 

1.28 

86 

0.88 

1.23 

86 

0.51 

1.17 

86 

0.57 

1.22 

86 

0.51 

0.99 

87 

0.51 

0.97 

87 

0.34 

0.79 

87 

0.38 

0.84 

87 

1.80 

1.73 

87 

2.09 

1.86 

87 

1.81 

1.77 

86 

1.94 

1.82 

86 

2.25 

1.70 

87 

2.59 

1.79 

87 

1.63 

1.66 

87 

1.68 

1.68 

87 

2.34 

1.51 

87 

2.47 

1.50 

87 

4.15 

1.13 

86 

4.40 

0.95 

86 

3.91 

1.30 

87 

4.16 

1.16 

87 

3.65 

1.45 

86 

4.01 

1.16 

86 

3.83 

1.30 

86 

4.13 

1.03 

86 

3.09 

1.47 

86 

3.37 

1.39 

86 

3.00 

1.53 

86 

3.22 

1.45 

86 

1.41 

1.73 

87 

1.47 

1.79 

87 

2.51 

1.75 

86 

2.71 

1.80 

86 

3.36 

1.46 

86 

3.64 

1.32 

86 

3.56 

1.47 

85 

3.99 

1.17 

85 
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Job  Activities 

Work  in  a  team 

Lead  a  team 

Support/advise  peers 

Support/advise  siiMrdi nates 

Coach  peers 

Coach  subordinates 

Hake  oral  reports  (to  individuals) 

Hake  oral  reports  (to  groups) 

Relay  oral  instructions 
Interview 

Record  information 

Write  brief  messages 

Write  longer  reports 

Honi tor/ interpret  verbal  messages 

Recall  verbal  information 

Honi tor/interpret  numerical  information 

Recall  miner i cal  information 

Honi tor/ interpret  figural  information 

Recall  figural  information 

Follow  oral  directions 

Follow  written  directions 

Judge  size  and  distance 

Judge  location 

Judge  paths  of  moving  objects 

Solve  electrical  system  problems 

Solve  mechanical  system  problems 

Solve  logistical  problems 

Solve  tactical  maneuver  problems 

Solve  adninistrative  problems 

Solve  leadership  problems 

Solve  medical  problems 

Solve  commirication  problems 

Operate  precision  hand-held  equipment 

Operate  hand-held  tools 

Operate  hand-held  power  equipment 

Operate  larger  power  equipment 

Operate  full  keyboard 

Operate  numeric  keyboard 

Adjust  control  device  using  one  linb 

Adj  control  device  using  mult  linbs 

Drive  tracked  vehicle 

Drive  heavy  wheeled  vehicle 

Drive  light  wheeled  vehicle 

Aim: stationary  target 

Aim:a»ving  target 

Walk  long  distances 

Run  short  distances 

Push,  pull,  lift  heavy  weights 

Throw  objects 

Sort,  fold,  feed  by  hand 

Hake  coordinated  movements 

Work  long  hours 

Work  under  adverse  conditions 


Frequency 

Core 

Technical 

HEAN 

STD 

N 

HEAN 

STD 

N 

3.32 

1.38 

50 

3.34 

1.32 

50 

2.08 

1.55 

50 

2.34 

1.64 

50 

2.44 

1.49 

50 

2.48 

1.54 

50 

2.26 

1.58 

50 

2.50 

1.58 

SO 

2.10 

1.39 

50 

2.30 

1.49 

50 

2.12 

1.62 

50 

2.38 

1.65 

50 

2.38 

1.37 

50 

2.54 

1.42 

50 

1.42 

1.18 

50 

1.64 

1.35 

SO 

2.52 

1.25 

50 

2.70 

1.33 

50 

1.08 

1.37 

50 

1.24 

1.52 

SO 

2.58 

1.36 

50 

2.82 

1.48 

49 

1.82 

1.38 

50 

1.94 

1.41 

50 

0.58 

0.97 

50 

0.70 

1.13 

50 

1.54 

1.59 

50 

1.74 

1.79 

50 

2.58 

1.57 

50 

2.68 

1.66 

50 

1.48 

1.25 

50 

1.78 

1.54 

50 

2.06 

1.35 

50 

2.36 

1.56 

50 

1.74 

1.66 

50 

2.12 

1.88 

50 

1.68 

1.38 

50 

2.08 

1.66 

50 

3.74 

1.07 

50 

3.70 

1.05 

50 

3.88 

0.98 

50 

3.94 

1.11 

50 

2.40 

1.26 

50 

2.52 

1.49 

SO 

2.45 

0.82 

49 

2.45 

1.21 

49 

2.68 

1.02 

50 

2.90 

1.25 

50 

3.22 

1.25 

50 

3.92 

1.26 

50 

3.62 

1.07 

50 

4.12 

1.00 

50 

1.56 

1.50 

50 

1.88 

1.67 

SO 

1.20 

1.18 

50 

1.26 

1.45 

50 

1.32 

1.53 

50 

1.26 

1.51 

50 

1.36 

1.59 

50 

1.44 

1.64 

50 

0.68 

1.15 

50 

0.66 

1.19 

50 

1.10 

1.16 

50 

1.24 

1.36 

50 

2.92 

1.52 

50 

3.52 

1.54 

50 

4.18 

1.00 

50 

4.38 

0.88 

SO 

3.62 

1.18 

50 

4.04 

1.03 

50 

1.82 

1.52 

50 

2.12 

1.65 

50 

0.76 

0.94 

50 

0.92 

1.14 

50 

0.56 

0.84 

50 

0.72 

1.11 

50 

2.88 

1.55 

50 

3.10 

1.59 

50 

3.02 

1.46 

50 

3.30 

1.56 

SO 

1.36 

1.35 

SO 

1.76 

1.70 

50 

2.80 

1.60 

50 

3.16 

1.71 

SO 

3.94 

1.11 

50 

4.02 

1.19 

50 

2.26 

1.21 

SO 

1.78 

1.71 

50 

1.70 

1.37 

50 

1.72 

1.76 

50 

2.38 

1.24 

50 

2.04 

1.63 

SO 

3.54 

1.07 

SO 

2.32 

1.77 

SO 

3.42 

0.97 

50 

3.26 

1.31 

50 

1.48 

1.37 

50 

1.12 

1.30 

50 

1.64 

1.59 

50 

1.76 

1.64 

50 

2.86 

1.28 

SO 

3.10 

1.43 

50 

3.22 

1.15 

50 

3.28 

1.39 

50 

2.94 

1.39 

50 

3.14 

1.58 

50 

Gen. 

Soldiering 

Overall  Job 

HEAN 

STD 

N 

HEAN 

STD 

N 

3.70 

1.27 

SO 

3.58 

1.21 

SO 

2.76 

1.62 

50 

2.64 

1.56 

50 

2.84 

1.61 

SO 

2.68 

1.52 

SO 

2.90 

1.57 

50 

2.74 

1.52 

50 

2.56 

1.58 

SO 

2.36 

1.47 

50 

2.66 

1.77 

50 

2.42 

1.63 

50 

2.68 

1.32 

50 

2.66 

1.30 

SO 

1.88 

1.45 

50 

1.68 

1.35 

50 

2.74 

1.31 

50 

2.64 

1.24 

50 

1.18 

1.37 

50 

1.22 

1.50 

50 

2.41 

1.26 

49 

2.67 

1.33 

49 

1.90 

1.28 

SO 

1.96 

1.32 

50 

0.80 

1.23 

50 

0.70 

1.16 

50 

1.76 

1.78 

SO 

1.74 

1.77 

50 

2.70 

1.67 

50 

2.74 

1.61 

50 

1.66 

1.44 

50 

1.72 

1.49 

50 

2.26 

1.51 

50 

2.20 

1.47 

50 

1.82 

1.56 

50 

1.94 

1.72 

50 

1.92 

1.43 

50 

1.96 

1.55 

50 

3.72 

1.01 

50 

3.88 

0.98 

SO 

3.74 

1.03 

SO 

3.94 

1.11 

50 

2.76 

1.33 

50 

2.76 

1.35 

50 

3.10 

1.10 

49 

2.98 

0.99 

49 

2.86 

1.01 

SO 

3.00 

1.05 

50 

2.10 

1.46 

50 

3.36 

1.26 

50 

2.36 

1.21 

50 

3.50 

1.20 

50 

1.48 

1.49 

50 

1.74 

1.58 

50 

1.82 

1.77 

50 

1.60 

1.58 

50 

1.30 

1.49 

50 

1.34 

1.52 

50 

1.76 

1.72 

50 

1.62 

1.60 

SO 

1.00 

1.56 

50 

0.90 

1.40 

50 

1.54 

1.58 

50 

1.40 

1.43 

50 

2.18 

1.30 

50 

2.94 

1.46 

SO 

2.54 

1.34 

SO 

3.80 

1.20 

50 

2.34 

1.35 

SO 

3.44 

1.33 

SO 

1.26 

1.37 

SO 

2.00 

1.77 

SO 

0.84 

1.22 

SO 

0.94 

1.22 

50 

0.56 

0.99 

50 

0.70 

1.13 

50 

2.08 

1.26 

SO 

2.68 

1.56 

SO 

2.24 

1.32 

49 

2.86 

1.51 

49 

1.56 

1.58 

50 

1.64 

1.56 

SO 

2.46 

1.45 

SO 

2.94 

1.60 

SO 

3.16 

1.25 

50 

3.74 

1.12 

50 

3.50 

1.52 

50 

3.00 

1.36 

SO 

3.00 

1.82 

SO 

2.56 

1.58 

50 

2.98 

1.45 

SO 

2.82 

1.37 

SO 

3.88 

1.06 

50 

3.52 

1.16 

SO 

3.20 

1.12 

50 

3.38 

1.01 

50 

1.86 

1.60 

SO 

1.68 

1.50 

SO 

1.52 

1.54 

SO 

1.70 

1.61 

SO 

2.66 

1.27 

SO 

2.98 

1.29 

50 

3.30 

1.37 

SO 

3.38 

1.19 

50 

2.98 

1.49 

SO 

3.14 

1.50 

SO 

JOB  ACTIVITY 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


M0S=71L 

Job  Activi ties 

Work  in  a  team 

Lead  a  team 

Support/ advise  peers 

Support/advise  s(i)ordi nates 

Coach  peers 

Coach  subordinates 

Hake  oral  reports  (to  individuals) 

Make  oral  reports  (to  groups) 

Relay  oral  instructions 
Interview 

Record  information 

Write  brief  messages 

Write  longer  reports 

Monitor/ interpret  verbal  messages 

Recall  verbal  information 

Monitor/ interpret  runerical  information 

Recall  nunerical  information 

Monitor/ interpret  figural  information 

Recall  figural  information 

Follow  oral  directions 

Follow  written  directions 

Judge  sizr  and  distance 

Judge  location 

Judge  paths  of  moving  objects 

Solve  electrical  system  problems 

Solve  mechanical  system  problems 

Solve  logistical  problems 

Solve  tactical  maneuver  problems 

Solve  administrative  problems 

Solve  leadership  problems 

Solve  medical  problems 

Solve  cornmnication  problems 

Operate  precision  hand-held  equipment 

Operate  hand-held  tools 

Operate  hand-held  power  equipment 

Operate  larger  power  eqpipment 

Operate  full  keyboard 

Operate  runeric  keyboard 

Adjust  control  device  using  one  linb 

Adj  control  device  using  mult  linte 

Drive  tracked  vehicle 

Drive  heavy  tiieelcd  vehicle 

Drive  light  wheeled  vehicle 

Aim:ststionary  target 

Aim:moving  target 

Walk  long  distances 

Run  short  distances 

Push,  pull,  lift  heavy  weights 

Throw  objects 

Sort,  fold,  feed  by  hand 

Make  coordinated  movemants 

Work  long  hours 

Work  leder  averse  conditions 


Frequency 

Core 

Technical 

MEAN 

STD 

M 

MEAN 

STD 

N 

2.94 

1.36 

52 

3.08 

1.38 

52 

1.67 

1.59 

52 

1.80 

1.70 

51 

2.65 

1.27 

52 

2.88 

1.31 

51 

2.17 

1.69 

52 

2.45 

1,76 

51 

2.38 

1.30 

52 

2.59 

1.30 

51 

2.12 

1.69 

52 

2.41 

1.73 

51 

2.62 

1.47 

52 

2.58 

1.47 

52 

1.69 

1.50 

52 

1.73 

1.54 

52 

2.79 

1.32 

52 

3.18 

1.20 

51 

1.73 

1.56 

52 

1.92 

1.62 

52 

3.52 

1.43 

52 

3.62 

1.30 

52 

3.04 

1.63 

52 

3.21 

1.39 

52 

1.44 

1.39 

52 

1.53 

1.36 

51 

2.54 

1.42 

52 

2.88 

1.53 

51 

2.96 

1.30 

52 

3.24 

1.27 

51 

1.83 

1.41 

52 

2.08 

1.56 

52 

1.98 

1.31 

52 

2.28 

1.53 

50 

1.25 

1.33 

52 

1.38 

1.36 

52 

1.15 

1.14 

52 

1.33 

1.38 

51 

3.90 

1.29 

52 

4.20 

1.11 

51 

4.12 

0.94 

52 

4.41 

0.90 

51 

1.13 

1.07 

52 

0.92 

1.15 

51 

1.33 

0.86 

52 

1,02 

1.15 

50 

0.79 

0.94 

52 

0.69 

1.03 

51 

0.60 

1,05 

52 

0.48 

0.87 

52 

0.75 

1,08 

52 

0.63 

0.99 

52 

0.71 

1.05 

52 

0.67 

1.17 

52 

0.40 

0.75 

52 

0.25 

0.62 

52 

3.00 

1.41 

52 

3.43 

1.50 

51 

1.52 

1.55 

52 

1.73 

1.79 

51 

0.38 

0,95 

52 

0.31 

0.85 

52 

1.52 

1.53 

52 

1.67 

1,70 

51 

0.27 

0.74 

52 

0.29 

0.87 

52 

0.71 

0.89 

52 

0.56 

0.87 

52 

0,37 

0.77 

52 

0.25 

0.62 

52 

0.27 

0.66 

52 

0.17 

0.51 

52 

4.58 

0.78 

52 

4.54 

0.93 

50 

2.16 

1.72 

51 

2.29 

1,79 

51 

1.00 

1.22 

51 

0.96 

1.22 

51 

1,00 

1.34 

51 

1.04 

1.34 

51 

0.19 

0,53 

52 

0.06 

0.31 

52 

0.48 

0,78 

52 

0.21 

0.54 

52 

1.62 

1.21 

52 

1.15 

1.27 

52 

1.65 

1.25 

52 

0.75 

1.18 

51 

1.00 

1.25 

52 

0.49 

0.99 

51 

1.46 

0.85 

52 

0.88 

0.95 

49 

2.71 

1.51 

52 

1.29 

1.45 

51 

1.46 

1.09 

52 

0.83 

0.96 

52 

0.69 

0.90 

52 

0.37 

0.69 

52 

2.58 

1.45 

52 

2.55 

1.55 

51 

1.33 

1.37 

52 

1.25 

1.48 

52 

2.31 

1.25 

52 

1.94 

1.45 

51 

1.40 

1.09 

52 

1.37 

1.40 

51 

Gen. 

Soldiering 

Overall  Job 

MEAN 

STO 

N 

MEAN 

STO 

N 

3.73 

1.29 

52 

3.52 

1.21 

52 

2.39 

1.78 

51 

2.37 

1.71 

51 

3.33 

1.16 

51 

3.24 

1.12 

51 

2.65 

1.80 

51 

2.57 

1.69 

51 

2.76 

1.42 

51 

2.69 

1.30 

51 

2.55 

1.84 

51 

2.53 

1.75 

51 

2.67 

1.41 

52 

2.75 

1.43 

52 

1.94 

1.66 

52 

2.10 

1.65 

52 

3.24 

1.41 

51 

3.31 

1.24 

51 

1.56 

1.61 

52 

1.87 

1.60 

52 

2.73 

1.56 

52 

3.42 

1.35 

52 

2.56 

1.55 

52 

3,13 

1.39 

52 

1.16 

1.29 

51 

1.57 

1.42 

51 

2.53 

1.47 

51 

2.80 

1.51 

51 

3.10 

1.32 

51 

3.22 

1.30 

51 

1.67 

1.46 

52 

2.02 

1.54 

52 

2.04 

1,50 

50 

2.36 

1.50 

50 

1.56 

1.63 

52 

1.58 

1.55 

52 

1.80 

1.76 

51 

1.63 

1.48 

51 

4.00 

1.30 

51 

4.20 

1.10 

51 

4.10 

1.15 

51 

4.37 

0.85 

51 

2.15 

1.78 

52 

1.67 

1.46 

52 

2.96 

1.56 

51 

2.28 

1.50 

50 

1.51 

1.69 

51 

1.16 

1.39 

51 

0.78 

1.32 

51 

0.61 

1.02 

51 

0.83 

1.26 

52 

0.75 

1.12 

52 

0.90 

1.36 

52 

0.75 

1.20 

52 

0.73 

1.34 

52 

0.52 

0.98 

52 

2.37 

1.67 

51 

3.20 

1.47 

51 

1.90 

1.85 

51 

1.92 

1.78 

51 

0.46 

1.09 

52 

0.44 

1.07 

52 

1.75 

1.83 

51 

1.78 

1.74 

51 

0.37 

0.93 

52 

0.31 

0.78 

52 

1.15 

1.38 

52 

0.92 

1.13 

52 

0.42 

0.82 

52 

0.31 

0.64 

52 

0.27 

0.72 

52 

0.23 

0.65 

52 

2.22 

1.74 

49 

4.02 

1.22 

50 

1.22 

1.52 

50 

2.18 

1.75 

51 

1.14 

1.40 

51 

1.20 

1.36 

51 

1.20 

1.54 

51 

1.24 

1.53 

51 

0.21 

0.64 

52 

0.13 

0.44 

52 

0.65 

1.20 

52 

0.48 

1.00 

52 

2.08 

1.61 

52 

1.75 

1.53 

52 

3,15 

1.83 

52 

2.58 

1.59 

52 

1.90 

1.99 

52 

1.69 

1.73 

52 

2.90 

1.55 

51 

2.33 

1.40 

51 

3.43 

1.39 

51 

2.90 

1.30 

51 

2.17 

1.59 

52 

1.90 

1.35 

52 

1.29 

1.60 

52 

1.13 

1.43 

52 

1.63 

1.55 

51 

2.45 

1.45 

51 

2.06 

1.69 

52 

1.88 

1.59 

52 

2.86 

1.46 

51 

2.63 

1.36 

51 

2.57 

1.85 

51 

2.27 

1.72 

51 

E-39 


JOB  ACTIVITY 

Frequency  and  Importance  Ratings  Descriptive  Statistics 


HOS^ALL 

Job  Activities 

Work  in  a  team 

Lead  a  team 

Support/advise  peers 

Support/advise  sifeordi nates 

Coach  peers 

Coach  subordinates 

Hake  oral  reports  (to  individuals) 

Hake  oral  reports  (to  groups) 

Relay  oral  instructions 
Interview 

Record  information 

Write  brief  messages 

Write  longer  reports 

Honi tor/ interpret  verbal  messages 

Recall  verbal  information 

Honi tor/ interpret  runerical  information 

Recall  nunerical  information 

Honi  tor/ interpret  figural  information 

Recall  figural  information 

Follow  oral  directions 

Follow  written  directions 

Judge  size  and  distance 

Judge  location 

Judge  paths  of  moving  objects 

Solve  electrical  system  problems 

Solve  mechanical  system  problems 

Solve  logistical  problems 

Solve  tactical  maneuver  problems 

Solve  adninistrative  problems 

Solve  leadership  problems 

Solve  medical  problems 

Solve  conmLnication  problems 

Operate  precision  hand-held  equipment 

Operate  hand-held  tools 

Operate  hand-held  power  equipment 

Operate  larger  power  equipment 

Operate  full  keyboard 

Operate  runeric  keyboard 

Adjust  control  device  using  or>e  linb 

Adj  control  device  using  mult  linbs 

Drive  tracked  vehicle 

Drive  heavy  wheeled  vehicle 

Drive  light  tiieeled  vehicle 

Aim:stationary  target 

Aimtffloving  target 

Walk  long  distances 

Run  short  distances 

Push,  pull,  lift  heavy  weights 

Throw  objects 

Sort,  fold,  feed  by  hand 

Hake  coordinated  movaments 

Work  long  hours 

Work  irder  adverse  conditions 


Frequency 

Core 

Techni 

cal 

HEAN 

STD 

N 

HEAN 

STD 

N 

3.22 

1.65 

190 

3.24 

1.80 

188 

3.14 

1.50 

190 

3.13 

1.84 

189 

1.S4 

1.87 

190 

2.04 

2.00 

190 

0.94 

1.27 

189 

1.13 

1.59 

189 

1.90 

1.90 

190 

1.66 

1.68 

190 

1.34 

1.58 

190 

1.60 

1.89 

189 

1.67 

1.62 

190 

1.76 

1.76 

189 

1.23 

1.68 

190 

1.34 

1.82 

188 

0.69 

1.29 

190 

0.74 

1.38 

189 

1.06 

1.51 

190 

1.20 

1.71 

189 

0.73 

1.28 

190 

0.80 

1.44 

190 

1.66 

1.47 

190 

1.53 

1.51 

189 

0.20 

0.71 

189 

0.20 

0.77 

189 

0.36 

0.78 

190 

0.43 

1.01 

190 

0.15 

0.54 

189 

0.18 

0.63 

189 

2.69 

1.51 

190 

2.70 

1.65 

188 

1.6B 

1.90 

190 

1.78 

1.98 

189 

0.23 

0.68 

190 

0.32 

0.94 

190 

0.47 

1.02 

189 

0.49 

1.10 

188 

1.46 

1.24 

190 

1.04 

1.27 

188 

1.15 

1.32 

190 

1.34 

1.57 

186 

0.29 

0.73 

190 

0.34 

0.90 

188 

0.55 

0.95 

190 

0.60 

1,15 

189 

0.18 

0.59 

190 

0.20 

0.71 

189 

0.12 

0.48 

190 

0.16 

0.69 

189 

0.20 

0.60 

190 

0.19 

0.68 

188 

0.07 

0.39 

190 

0.09 

0.53 

189 

1.28 

1.40 

190 

1.32 

1.57 

189 

0.06 

0.37 

190 

0.05 

0.38 

189 

0.59 

0.99 

190 

0.55 

1.08 

188 

1.44 

1.65 

190 

1.18 

1.59 

188 

1,36 

1.60 

190 

1.46 

1.75 

188 

0.09 

0.51 

190 

0.09 

0.49 

189 

1.15 

1.61 

190 

1.11 

1.65 

189 

0,27 

0.88 

189 

0.25 

0.88 

189 

0,13 

0.67 

190 

0.13 

0.69 

190 

0,72 

1.34 

189 

0.74 

1.48 

187 

0,19 

0.71 

189 

0.22 

0.83 

188 

2.13 

1.71 

190 

2.06 

1.97 

188 

0.18 

0,67 

190 

0.17 

0.70 

189 

0.57 

1.16 

190 

0.68 

1.40 

190 

o.a 

1.05 

190 

0.49 

1.17 

189 

1.83 

2.05 

190 

1,75 

2.07 

189 

2.27 

1.88 

190 

2.26 

1.94 

189 

2.20 

1.95 

190 

2.12 

1.91 

189 

1.65 

1.64 

190 

1.66 

1.63 

189 

2.86 

1.65 

190 

2.86 

1.92 

188 

3.74 

1.18 

190 

3.93 

1.15 

189 

3.09 

1.49 

190 

3.ir 

1.77 

188 

2.59 

1.52 

189 

2.74 

1.90 

187 

2.66 

1.59 

190 

2.84 

1.80 

188 

0.63 

1.29 

190 

0.54 

1.19 

189 

2.72 

1.45 

189 

2.88 

1.47 

186 

E-40 


Gen. 

Soldiering 

Overall  Job 

HEAN 

STD 

N 

HEAN 

STD 

N 

3.30 

1.56 

188 

3.47 

1.57 

187 

3.90 

1.30 

189 

3.85 

1.31 

189 

1.65 

1.65 

190 

2.00 

1.92 

190 

1.20 

1.58 

189 

1.26 

1.66 

189 

1.66 

1.68 

190 

2.02 

1.92 

190 

1.61 

1.79 

190 

1.65 

1.81 

189 

1.76 

1.70 

189 

1.85 

1.72 

189 

0.88 

1.33 

188 

1.22 

1.67 

188 

0.58 

1.14 

189 

0.73 

1.36 

189 

0.78 

1.24 

189 

1,12 

1.59 

189 

0.58 

1.16 

190 

0.75 

1.34 

190 

1.78 

1.53 

189 

1.74 

1.51 

189 

0.24 

0.85 

189 

0.23 

0.84 

189 

0.47 

1.07 

190 

0.45 

1.05 

190 

0.14 

0.58 

189 

0.15 

0.59 

189 

2.89 

1.49 

189 

2.99 

1.57 

189 

1.57 

1.81 

190 

1.73 

1.92 

190 

0.27 

0.85 

190 

0.28 

0.83 

190 

0.38 

0.97 

188 

0.48 

1.09 

188 

1.19 

1.27 

188 

1.19 

1.27 

187 

1.34 

1.53 

186 

1.35 

1.53 

186 

0.25 

0.74 

189 

0.33 

0.88 

189 

0.35 

0.79 

189 

0.58 

1.09 

189 

0.23 

0.75 

190 

0.24 

0.75 

190 

0.11 

0.52 

189 

0.16 

0.68 

189 

0,23 

0.72 

189 

0.23 

0.73 

189 

0.10 

0.55 

190 

0.11 

0.59 

190 

1.26 

1.42 

190 

1.36 

1.54 

190 

0.08 

0.55 

190 

0.08 

0.55 

190 

0.65 

1.13 

189 

0.67 

1.14 

189 

0.93 

1.30 

189 

1.28 

1.56 

189 

1.26 

1.57 

189 

1.49 

1.74 

188 

0.12 

0.58 

190 

0.12 

0.61 

190 

0.80 

1.32 

189 

1.19 

1.68 

190 

0.29 

0.95 

189 

0.29 

0.95 

189 

0.12 

0.66 

190 

0.13 

0.69 

190 

0.85 

1.51 

187 

0.81 

1.47 

187 

0.28 

0.96 

188 

0.25 

0.91 

188 

2.37 

1.81 

189 

2.37 

1.81 

189 

0.22 

0.82 

190 

0.18 

0.72 

190 

0.69 

1.35 

190 

0.64 

1.29 

190 

0.49 

1.14 

189 

0.44 

1.07 

189 

1.08 

1.48 

189 

1.64 

1.92 

190 

1.56 

1.58 

189 

2.10 

1.80 

188 

1.49 

1.57 

189 

2.06 

1.84 

189 

1,42 

1.54 

189 

1.74 

1.64 

189 

3.07 

1.58 

189 

3.06 

1.60 

189 

3.70 

1.17 

189 

3.87 

1.12 

189 

3,76 

1.21 

189 

3.62 

1.28 

189 

3.25 

1.56 

188 

3.11 

1.54 

188 

3.05 

1.61 

189 

3.09 

1.55 

189 

O.U 

1.10 

189 

0.60 

1.24 

189 

2.80 

1.54 

187 

2.97 

1.48 

187 

Tables  of  Component  Rating  Means,  Standard  Deviations,  and  N 
Ordered  by  CTP  Mean  Rating 

Task  Category  Instrument 
IIB 
63B 
71L 


Job  Activity  Instrument 
IIB 
63B 
7IL 
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TASK  CATEGORY 

Frequency  and  Iirportance  Ratings  Descriptive  Statistics 

Ordered  by  Core  Technical  Rating  Means 

MOS^IIB 


Frequency 

Core  Technical 

Gen.  Soldieri 

ing 

Overall  Job 

Task  Categories 

MEAN 

STD 

N 

MeAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Fire  individual  weapons 

4.19 

1.02 

88 

4.67 

0.56 

87 

4.44 

0.96 

87 

4.52 

0.76 

87 

Protect  against  NBC  hazards 

3.81 

1.10 

88 

4.63 

0.55 

86 

4.35 

0.88 

86 

4.42 

0.87 

86 

Hove  and  react  in  the  field 

4.27 

1.00 

88 

4.60 

0.72 

87 

4.06 

1.21 

87 

4.38 

0.93 

87 

Perf  operator  chks  &  services  on  weapons 

4.24 

0.90 

88 

4.50 

0.86 

88 

4.19 

1.12 

88 

4.41 

0.87 

88 

Use  maps 

4.02 

1.11 

88 

4.45 

0.82 

87 

3.98 

1.13 

87 

4.10 

1.07 

87 

Give  first  aid 

3.41 

1.17 

88 

4.35 

0.96 

86 

4.28 

0.93 

86 

4.31 

0.80 

86 

Survive  in  the  field 

4.05 

1.29 

88 

4.33 

1.14 

87 

3.98 

1.39 

86 

4.16 

1.25 

87 

Navigate 

3.68 

1.27 

88 

4.30 

1.06 

87 

3.90 

1.29 

87 

4.05 

1.17 

87 

Use  hand  and  arm  signals 

4.10 

0.98 

88 

4.28 

1.10 

87 

3.82 

1.28 

87 

4.05 

1.12 

87 

Detect  and  identify  targets 

3.64 

1.19 

88 

4.28 

0.95 

87 

3.86 

1.18 

87 

4,05 

1.07 

87 

Use  hand  grenades 

3.38 

1.23 

88 

4.22 

1.07 

86 

3.80 

1.28 

86 

4.06 

1.06 

86 

Give  short  oral  reports 

3.77 

1.14 

88 

4.21 

1.08 

87 

3.74 

1.32 

87 

3.97 

1.09 

87 

Serd  and  receive  radio  messages 

3.58 

1.14 

88 

4.20 

0.96 

87 

3.78 

1.14 

87 

3.90 

1.05 

87 

Perf  operator  mint  chks  and  services 

3.75 

1.42 

88 

3.89 

1.41 

88 

3.56 

1.46 

87 

3.87 

1.37 

86 

Read  tech/field  mnuals,  reg  etc 

3.67 

1.10 

88 

3.26 

1.14 

87 

3.80 

1.18 

87 

3.85 

1.14 

87 

Sketch  maps,  overlays,  or  range  cards 

3.33 

1.45 

88 

3.61 

1.53 

87 

3.09 

1.59 

87 

3.41 

1.55 

87 

Customs  and  laws  of  war 

3.03 

1.43 

88 

3.54 

1.44 

85 

3.48 

1.36 

86 

3.47 

1.39 

86 

Place  &  camoufl  tactical  equip  and  mat 

2.97 

1.73 

88 

3.51 

1.74 

88 

3.10 

1.67 

88 

3.38 

1.72 

88 

Engage  in  hand-to-hand  cotrbat 

2.31 

1.49 

88 

3.38 

1.66 

87 

3.07 

1.67 

87 

3.22 

1.57 

87 

Handle  demolitions  or  mines 

2.42 

1.53 

88 

3.34 

1.72 

87 

2.83 

1.67 

87 

3.07 

1.70 

87 

Control  individuals  and  crowds 

2.97 

1.41 

88 

3.34 

1.50 

87 

3.11 

1.60 

87 

3.38 

1.47 

87 

Act  as  a  model 

2.78 

1.80 

88 

3.15 

1.95 

86 

3.09 

1.90 

86 

3.17 

1.93 

86 

Comnnicate 

2.75 

1.95 

88 

3.14 

2.04 

87 

2.91 

2.02 

87 

3.07 

2.03 

86 

Give  directions  ard  instructions 

2.78 

1.56 

87 

3.07 

1.66 

86 

2,87 

1.68 

86 

2.99 

1.70 

86 

Lead 

2.56 

1.79 

88 

3.00 

1.99 

86 

2.84 

1.96 

86 

3.02 

1.97 

86 

Troubleshoot  weapons 

2.26 

1.62 

88 

2.82 

1.82 

88 

2.51 

1,72 

88 

2.65 

1.71 

88 

Decode  data 

2.15 

1.44 

88 

2.80 

1.70 

86 

2.53 

1.61 

86 

2.52 

1.66 

85 

Operate  track  vehicles 

2.74 

2.01 

88 

2.77 

2.02 

88 

2.43 

1.85 

88 

2.67 

1.95 

88 

Direct/ lead  teams 

2.10 

1.82 

88 

2.70 

2.10 

87 

2.28 

1.93 

87 

2.41 

2.02 

87 

Monitor/inspect 

2.35 

1.88 

88 

2.69 

1.98 

87 

2.57 

1.94 

87 

2.68 

2.00 

87 

Operate  wheeled  vehicles 

2.53 

1.52 

88 

2.68 

1.43 

87 

2.57 

1.48 

87 

2.74 

1.53 

87 

Coinsel 

2.11 

1.88 

88 

2.53 

2.07 

87 

2.46 

2.04 

87 

2.53 

2.10 

87 

Install  electronic  ccmponents 

2.33 

1.72 

88 

2.45 

1.84 

88 

2.33 

1.78 

88 

2.35 

1.81 

88 

Train 

2.03 

1.96 

88 

2.45 

2.18 

87 

2.33 

2.09 

87 

2.41 

2.17 

87 

Plan  operations 

1.67 

1.74 

88 

2.22 

2.09 

87 

2.06 

2.01 

87 

2.11 

2.05 

87 

Plan  placement/usa  of  tactical  equip 

1.78 

1.85 

88 

2.12 

2.08 

86 

1.77 

1.85 

86 

1.99 

1.94 

86 

Pack  and  load  mterials 

2.03 

1.58 

88 

2.10 

1.58 

87 

1.93 

1.52 

87 

1.95 

1.55 

87 

Fire  heavy  direct  fire  weapons 

1.58 

1.80 

88 

2.03 

2.12 

88 

1.65 

1.89 

88 

1.86 

2.02 

88 

Troubleshoot  mechanical  systems 

1.72 

1.75 

88 

1.92 

1.84 

88 

1.74 

1.73 

88 

1.93 

1.88 

88 

Repair  weapons 

1.40 

1.38 

87 

1.86 

1.75 

87 

1.69 

1.68 

87 

1.85 

1.76 

87 

Install  wire  and  cables 

1.52 

1,49 

88 

1.84 

1.70 

86 

1.65 

1.62 

85 

1.66 

1.64 

85 

Repair  mechanical  systems 

1.64 

1.61 

88 

1.80 

1.73 

88 

1.80 

1.73 

88 

1.93 

1.83 

88 

Personnel  Adninstration 

1.58 

1.94 

88 

1.66 

2.02 

87 

1.59 

1.96 

87 

1.64 

2.01 

87 

Operate  electronic  equipment 

1.27 

1.64 

88 

1.41 

1.76 

88 

1.26 

1.64 

88 

1.36 

1.78 

87 

Analyte  in'eUigance  data 

1.13 

1.53 

88 

1.40 

1.76 

86 

1.22 

1.59 

86 

1.28 

1.64 

86 

Provide  cocnseling 

1.18 

1.63 

88 

1.28 

1.78 

87 

1.26 

1.70 

87 

1,32 

1.86 

87 

Order  equippnant  and  supplies 

1.02 

1.40 

88 

l.b 

1.74 

86 

1.15 

1.62 

86 

1.13 

1.60 

86 

Oetermina  firing  data-indirect  weapons 

1.05 

1.55 

88 

1.24 

1.79 

86 

1,05 

1.60 

86 

1.14 

1.70 

86 

E-42 
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TASK  CATEGORY  (Contd.) 

Frequency  and  Importance  Ratings  Descriptive  Statistics 

Ordered  by  Core  Technical  Rating  Means 

MOS^IIB 

Frequency  Core  Technical  Gen.  Soldiering  Overall  Job 


Task  Categories 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Analyze  weather  conditions 

0.90 

1.41 

88 

1.14 

1.82 

87 

0.90 

1.49 

87 

0.97 

1.62 

87 

Draw  maps  and  overlays 

0.89 

1.43 

68 

1.13 

1.71 

88 

0.95 

1.50 

88 

0.98 

1.52 

88 

Paint 

7.70 

1.23 

88 

1.07 

1.20 

86 

1.27 

1.22 

86 

1.22 

1.18 

86 

Conduct  land  surveys 

0.94 

1.65 

87 

1.02 

1.76 

86 

0.87 

1.56 

86 

0.90 

1.59 

86 

Prepare  technical  forms  and  docunents 

1.03 

1.41 

88 

0.99 

1.53 

87 

1.02 

1.52 

87 

0.95 

1.45 

87 

Receive, store, tissue  sif)p,  equip,  etc 

0.91 

1.37 

88 

0.92 

1.40 

87 

0.94 

1.43 

87 

1.01 

1.47 

87 

Fire  indirect  fire  weapons 

0.81 

1.53 

88 

0.84 

1.63 

87 

0.86 

1.66 

87 

0.85 

1.64 

87 

Record,  file,  and  dispatch  information 

0.91 

1.28 

88 

0.80 

1.27 

87 

0.90 

1.35 

87 

0.89 

1.34 

87 

Control  money 

0.78 

1.44 

88 

0.80 

1.58 

87 

0.82 

1.57 

87 

0.86 

1.62 

87 

Write  docunents  and  correspondence 

0.67 

1.25 

88 

0.80 

1.52 

88 

0.78 

1.53 

88 

0.70 

1.37 

88 

Load  &  unload  field  artillery/tank  guis 

0.62 

1.39 

87 

0.71 

1.58 

87 

0.60 

1.35 

87 

0.68 

1.50 

87 

Prepare  heavy  weapons  for  tactical  use 

0.60 

1.37 

88 

0.70 

1.57 

88 

0.60 

1.39 

C3 

0.63 

1.45 

88 

Prepare  equip  and  supplies  for  air  drop 

0.57 

0.99 

88 

0.69 

1.30 

88 

0.72 

1.33 

88 

0.72 

1.35 

88 

Use  audiovisual  equipment 

0.63 

0.97 

88 

0.66 

1.18 

87 

0.72 

1.16 

87 

0.71 

1.18 

87 

Draw  illustrations 

0.56 

1.13 

88 

0.66 

1.35 

87 

0.61 

1.25 

87 

0.54 

1.17 

87 

Write  and  deliver  presentations 

0.58 

1.17 

88 

0.66 

1.40 

87 

0.61 

1.24 

87 

0.64 

1.34 

87 

Interview 

0.53 

1.18 

88 

0.61 

1.31 

88 

0.60 

1.32 

88 

0.61 

1.31 

88 

Inspect  electronic  systems 

0.57 

1.15 

88  , 

,  0.61 

1.27 

87 

0.52 

1.09 

87 

0.55 

1.18 

87 

Operate  boats 

0.45 

0.92 

88 

0.60 

1.19 

88 

0.49 

1.05 

88 

0.52 

1.02 

88 

Operate  gas  t  electric  powered  equip 

0.64 

0.98 

88 

0.60 

1.01 

88 

0.61 

1.02 

88 

0.64 

1.08 

88 

inspect  electrical  systems 

0.49 

1.11 

88 

0.57 

1.30 

87 

0.52 

1.15 

87 

0.52 

1.15 

87 

Type 

0.70 

1.10 

88 

0.55 

0.99 

88 

0.68 

1.23 

87 

0.52 

0.97 

88 

Reproduce  printed  material 

0.86 

1.27 

88 

0.53 

0.95 

87 

0.74 

1.19 

87 

0.70 

1.11 

87 

Conpute  statistics/other  math 

0.48 

1.05 

88 

0.52 

1.21 

86 

0.52 

1.15 

86 

0.52 

1.17 

86 

Analyze  electronic  signals 

0.33 

0.88 

88 

0.50 

1.22 

88 

0.41 

1.02 

88 

0.43 

1.09 

88 

Repair  electrical  systems 

0.45 

0.99 

88 

0.47 

1.08 

87 

0.48 

1.06 

87 

0.54 

1.15 

87 

Estimate  time  &  cost  of  maint  ops 

0.55 

1.41 

88 

0.45 

1.28 

86 

0.34 

1.11 

85 

0.41 

1.18 

85 

Repair  electronic  conponents 

0.39 

0.94 

88 

0.40 

1.02 

88 

0.41 

1.00 

88 

0.42 

0.98 

88 

Prepare  parachutes 

0.34 

0.94 

87 

0.36 

1.00 

87 

0.37 

0.98 

87 

0.37 

1.00 

87 

Cook 

0.40 

1.13 

87 

0.36 

1.05 

87 

0.38 

1.11 

87 

0.39 

1.07 

87 

Operate  computer  harcbsre 

0.40 

0.85 

88 

0.35 

0.88 

88 

0.47 

1.04 

88 

0.50 

1.13 

88 

Provide  medical  or  dental  treatment 

0.26 

0.84 

87 

0.33 

0.96 

86 

0.34 

1.01 

86 

0.30 

0.98 

86 

Receive  clients,  patients,  guests 

0.32 

0.89 

88 

0.28 

0.84 

87 

0.30 

0.85 

87 

0.31 

0.89 

87 

Assemble  steel  structures 

0.22 

0.69 

88 

0.26 

0.80 

88 

0.26 

0.80 

88 

0.27 

0.81 

88 

Provide  prograniaing  t  DP  sipport 

0.22 

0.65 

88 

0.26 

0.91 

86 

0.23 

0.79 

86 

0.17 

0.62 

86 

Control  air  traffic 

0.16 

0.57 

88 

0.25 

0.86 

88 

0.18 

0.64 

88 

0.20 

0.76 

88 

Construct  wooden  bldgs  i  other  struct 

0.28 

0.79 

88 

0.24 

0.75 

87 

0.29 

0.78 

87 

0.28 

0.80 

87 

Translate  foreign  languages 

0.25 

0.86 

88 

0.21 

0.78 

87 

0.26 

0.96 

87 

0.22 

0.78 

87 

Perform  medical  laboratory  procedures 

0.18 

0.77 

88 

0.18 

0.81 

88 

0.17 

0.75 

88 

0.18 

0.81 

88 

Produce  technical  drawings 

0.15 

0.60 

88 

0.17 

0.68 

88 

0.17 

0.73 

88 

0.15 

0.65 

88 

Operate  power  excavating  equipment 

0.17 

0.61 

87 

0.16 

0.55 

87 

0.14 

0.55 

87 

0.16 

0.59 

87 

Repair  plastic  and  fiberglass 

0.22 

0.73 

88 

0.16 

0.57 

88 

0.18 

0.60 

88 

0.13 

0.42 

88 

Repair  metal 

0.23 

0.66 

88 

0.16 

0.52 

88 

0.20 

0.65 

88 

0.24 

0.69 

88 

Install  pipe  asseablies 

0.13 

0.60 

88 

0.15 

0.67 

88 

0.14 

0.59 

88 

0.15 

0.67 

88 

Operate  lifting, loading,lgrsding  equip 

0.15 

0.67 

88 

0.11 

0.56 

87 

0.10 

0.48 

87 

0.10 

0.48 

87 

Construct  masonry  bldgs  t  structures 

0.10 

0.50 

88 

0.10 

0.50 

88 

0.10 

0.50 

88 

0.11 

0.60 

88 

Operate  radar 

0.13 

0.64 

88 

0.10 

0.50 

88 

0.11 

0.53 

88 

0.11 

0.60 

88 

Select, layout,tc lean  med/dantal  equip 

0.07 

0.45 

88 

0.06 

0.35 

88 

0.07 

0.45 

88 

0.07 

0.45 

88 

E-43 


TASK  CATEGORY 

Frecjuency  and  Importance  Ratings  Descriptive  Statistics 

Tlrdered  by  Core  Technical  Rating  Hearts 

HOS=63B 


Frequency 


Task  Categories 

MEAN 

STD 

M 

Read  tech/field  manuals,  reg  etc 

4.10 

1.05 

50 

Perf  operator  maint  chks  and  services 

4.02 

1.06 

50 

Operate  titeeled  vehicles 

3.92 

0.94 

50 

Troubleshoot  mechanical  systems 

3.66 

1.27 

50 

Inspect  electrical  systems 

3.38 

1.23 

50 

Repair  electrical  systems 

2.96 

1.26 

50 

Operate  gas  &  electric  powered  equip 

2.78 

1.27 

50 

Act  as  a  model 

2.64 

1.44 

50 

Repair  mechanical  systems 

4.12 

0.94 

50 

Use  maps 

2.72 

1.29 

50 

Perf  operator  chks  &  services  on  weapons 

2.64 

1.14 

50 

Use  hand  and  arm  signals 

2.56 

1.28 

50 

Prepare  technical  forms  artd  dociments 

2.48 

1.52 

50 

Give  directions  and  instructions 

2.34 

1.32 

50 

Coninxiicate 

2.32 

1.56 

50 

Give  first  aid 

2.60 

1.11 

50 

Order  equipment  and  supplies 

1.94 

1.46 

50 

Protect  against  NSC  hazards 

2.86 

1.13 

50 

Fire  individual  weapons 

2.84 

1.17 

50 

Lead 

2.04 

1.64 

50 

Record,  file,  and  dispatch  information 

2.12 

1.61 

50 

Repair  electronic  components 

1.78 

1.61 

50 

Send  and  receive  radio  messages 

2.24 

1.27 

50 

Receive, store, tissue  si^ip,  equip,  etc 

1.74 

1.48 

50 

Rep^r  metal 

1.56 

1.09 

50 

Give  short  oral  reports 

2.06 

1.36 

50 

Survive  in  the  field 

2.26 

1.38 

50 

Train 

1.56 

1.69 

50 

Monitor/ inspect 

1.70 

1.57 

50 

Paint 

1.66 

1.32 

50 

Operate  track  vehicles 

1.40 

1.43 

50 

Place  t  camoufl  tactical  equip  and  mat 

1.86 

1.40 

50 

Move  and  react  in  the  field 

1.94 

1.24 

50 

Estimate  time  1  cost  of  maint  ops 

1.56 

1.54 

50 

Navigate 

1.88 

1.21 

50 

Install  electronic  components 

1.36 

1.35 

50 

Counsel 

1.52 

1.55 

50 

Operate  lifting, loading, tgrading  equip 

1.42 

1.36 

50 

Customs  and  laws  of  war 

1.86 

1.26 

50 

Inspect  electronic  systems 

1.34 

1.70 

50 

Detect  and  identify  targets 

1.58 

1.30 

50 

Pack  and  load  materials 

1.68 

1.33 

50 

Install  wire  and  cables 

1.20 

1.16 

50 

Use  hand  grenades 

1.56 

1.30 

50 

Plan  placement/use  of  tactical  equip 

1.22 

1.43 

50 

Control  individuals  and  crowds 

1.38 

1.38 

50 

Personnel  Adainatration 

1.00 

1.54 

50 

Operate  electronic  equipment 

0.92 

1.12 

50 

Core  Technical 

Gen.  Soldiering 

Overall  Job 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

4.36 

0.80 

50 

3.76 

1.02 

50 

4.14 

0.90 

50 

4.18 

1.06 

50 

3.78 

1.15 

50 

4.16 

0.93 

50 

4.16 

0.84 

50 

3.90 

0.95 

50 

4.20 

0.90 

50 

4.10 

1.16 

50 

2.64 

1.24 

50 

3.72 

1.16 

50 

3.60 

1.26 

50 

1.98 

1.29 

50 

3.16 

1.31 

50 

3.46 

1.30 

50 

1.84 

1.36 

50 

3.02 

1.36 

50 

3.14 

1.32 

50 

2.34 

1.35 

50 

2.90 

1.36 

50 

2.92 

1.67 

49 

3.45 

1.54 

49 

3.24 

1.52 

49 

2.78 

1.30 

50 

2.78 

1.30 

50 

3.90 

0.99 

50 

2.78 

1.52 

50 

3.64 

1.12 

50 

3.54 

1.01 

50 

2.68 

1.68 

50 

3.84 

1.20 

50 

3.76 

1.25 

50 

2.66 

1.55 

50 

2.74 

1.26 

50 

2.82 

1.27 

50 

2.64 

1.68 

50 

2.02 

1.42 

50 

2.46 

1.54 

50 

2.61 

1.27 

49 

2.71 

1.35 

49 

2.69 

1.33 

49 

2.56 

1.55 

50 

2.96 

1.62 

50 

2.88 

1.59 

50 

2.50 

1.79 

50 

4.04 

0.92 

50 

4.00 

0.86 

50 

2.44 

1.69 

50 

1.76 

1.51 

50 

2.28 

1.63 

50 

2.36 

1.78 

50 

4.20 

0.90 

50 

3.88 

1.00 

50 

2.34 

1.92 

50 

4.18 

0.94 

50 

4.02 

0.96 

50 

2.22 

1.76 

49 

2.63 

1.93 

49 

2.49 

1.83 

49 

2.18 

1.60 

50 

1.82 

1.52 

50 

2.16 

1.61 

SO 

2.16 

1.80 

50 

1.26 

1.48 

50 

1.86 

1.74 

50 

2.10 

1.69 

50 

3.18 

1.57 

50 

2.94 

1.45 

50 

2.02 

1.67 

50 

1.58 

1.53 

50 

2.00 

1.68 

50 

1.94 

1.42 

50 

0.88 

1.08 

50 

1.68 

1.38 

50 

1.94 

1.52 

50 

2.54 

1.61 

50 

2.50 

1.46 

50 

1.88 

1.78 

50 

3.52 

1.61 

50 

3.34 

1.52 

SO 

1.88 

2.00 

50 

2.00 

1.99 

50 

1.98 

2.06 

50 

1.86 

1.68 

50 

1.84 

1.72 

50 

1.90 

1.68 

50 

1.84 

1.43 

50 

1.68 

1.53 

50 

1.82 

1.48 

49 

1.80 

1.69 

50 

1.50 

1.64 

50 

1.70 

1.72 

SO 

1.80 

1.70 

50 

2.72 

1.78 

50 

2.48 

1.69 

50 

1.78 

1.69 

50 

3.32 

1.43 

50 

3.16 

1.46 

50 

1.76 

1.66 

50 

1.12 

1.49 

50 

1.58 

1.60 

50 

1.76 

1.62 

50 

3.10 

1.59 

50 

2.90 

1.57 

50 

1.59 

1.61 

49 

1.43 

1.51 

49 

1.78 

1.57 

49 

1.58 

1.72 

50 

2.00 

1.90 

50 

1.84 

1,74 

50 

1.56 

1.50 

50 

1.16 

1.42 

50 

1.54 

1.49 

50 

1.54 

1.62 

50 

3.14 

1.57 

SO 

2.84 

1.56 

50 

1.50 

1.78 

50 

0.94 

1.35 

SO 

1.44 

1.75 

50 

1.46 

1.61 

50 

2.82 

1.67 

50 

2.62 

1.61 

50 

1.44 

1.39 

50 

1.92 

1.50 

50 

1.84 

1.45 

50 

1.41 

1.48 

49 

1.47 

1.43 

49 

1.59 

1.50 

49 

1.39 

1.63 

49 

2.92 

1.80 

49 

2.61 

1.62 

49 

1.14 

1.53 

50 

1.56 

1.69 

50 

1.48 

1.62 

50 

1.14 

1.47 

50 

2.00 

1.93 

50 

1.78 

1.72 

50 

1.12 

1.72 

50 

1.26 

1.79 

50 

1.24 

1.78 

SO 

1.06 

1.43 

49 

1.04 

1.37 

49 

1.10 

1.42 

49 

TASK  CATEGORT  (Contd.) 

Frequency  and  Importance  Ratings  Descriptive  Statistics 

Ordered  by  Core  Technical  Rating  Means 

HOS=63B 


Frequency 

Core  Technical 

Gen.  Soldiering 

Overall  Job 

Task  Categories 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Sketch  maps,  overlays,  or  range  cards 

1.38 

1.26 

50 

1.04 

1.38 

SO 

2.08 

1.82 

SO 

1.78 

1.59 

50 

Engage  in  hand-to-hand  combat 

1.12 

1.04 

so 

1.04 

1.34 

SO 

2.30 

1.72 

50 

1.94 

1.58 

50 

Repair  plastic  and  fiberglass 

0.66 

0.89 

50 

0.98 

1.39 

49 

0.55 

1.12 

49 

0.96 

1.43 

49 

Conduct  land  surveys 

0.80 

1.16 

50 

0.90 

1.47 

50 

1,12 

1.62 

50 

1.10 

1.61 

50 

Troubleshoot  weapons 

0.90 

1.28 

50 

0.90 

1.50 

49 

1.20 

1.62 

50 

1.14 

1.62 

49 

Decode  data 

0.96 

1.32 

50 

0.88 

1.36 

50 

1.44 

1.74 

50 

1.26 

1.58 

50 

Direct/lead  teams 

0.64 

0.96 

50 

0.88 

1.38 

50 

1.18 

1.67 

50 

1.16 

1.65 

50 

Repair  weapons 

0.84 

1.27 

50 

0.86 

1.46 

SO 

1.04 

1.50 

so 

1.10 

1.62 

50 

Handle  demolitions  or  mines 

0.80 

0.93 

50 

0.84 

1.40 

50 

1.50 

1.71 

SO 

1.42 

1.59 

50 

Type 

0.74 

1.01 

SO 

0.80 

1.21 

50 

0.88 

1.30 

SO 

0.94 

1.22 

50 

Reproduce  printed  material 

0.80 

1.14 

50 

0.78 

1.25 

50 

0.84 

1.25 

50 

0.88 

1.26 

50 

Provide  coinseling 

0.68 

1.30 

50 

0.70 

1.31 

50 

0.78 

1.36 

SO 

0.78 

1.39 

50 

Plan  operations 

0.70 

1.33 

50 

0.70 

1.34 

SO 

0.82 

1.53 

SO 

0.88 

1.56 

50 

Interview 

0.48 

0.95 

50 

0.66 

1.30 

so 

0.66 

1.26 

50 

0.68 

1.28 

50 

Operate  computer  hardware 

0.48 

0.86 

50 

0.64 

1.31 

50 

0.60 

1.09 

50 

0.66 

1.22 

50 

Write  docunents  and  correspondence 

0.60 

1.20 

50 

0.58 

1.26 

50 

0.66 

1.30 

50 

0.66 

1.33 

50 

Conpute  statistics/other  math 

0.38 

0.90 

50 

0.58 

1.28 

SO 

0,50 

1.15 

50 

0.56 

1.20 

50 

Draw  illustrations 

0.52 

1.25 

50 

0.54 

1.27 

SO 

0.54 

1.25 

50 

0.54 

1.27 

SO 

Write  and  deliver  presentations 

0.34 

0.75 

50 

0.52 

1.16 

50 

0.60 

1.25 

50 

0.58 

1.26 

50 

Draw  maps  and  overlays 

0.40 

0.97 

50 

0.48 

1.16 

50 

0.68 

1.43 

50 

0.52 

1.22 

50 

Analyze  intelligence  data 

0.50 

0.97 

50 

0.46 

0.97 

50 

0.64 

1.26 

SO 

0.68 

1.35 

50 

Analyze  weather  conditions 

0.42 

0.91 

50 

0.44 

1.03 

50 

0.52 

1.07 

50 

0.48 

1.05 

50 

Control  money 

0.56 

1.47 

50 

0.42 

1.21 

50 

0.58 

1.50 

50 

0.56 

1.49 

50 

Determine  firing  data-indirect  weapons 

0.40 

1.03 

50 

0.42 

1.18 

50 

0.60 

1.40 

50 

0.56 

1.33 

50 

Operate  power  excavating  equipment 

0.26 

0.63 

50 

0.38 

0.95 

50 

0.24 

0.74 

50 

0.24 

0.74 

50 

Use  audiovisual  equipment 

0.40 

0.83 

50 

0.38 

0.95 

50 

0.54 

1.16 

50 

0.48 

1.03 

50 

Install  pipe  assemblies 

0.18 

0.44 

50 

0.34 

0.98 

50 

0.14 

0.61 

50 

0.32 

0.96 

50 

Provide  progranming  i  OP  support 

0.26 

0.75 

50 

0.33 

1.11 

49 

0.29 

0.91 

49 

0.35 

1.11 

49 

Prepare  equip  and  supplies  for  air  drop 

0.24 

0.56 

50 

0.32 

0.79 

50 

0.34 

0.85 

50 

0.32 

0.79 

50 

Cook 

0.22 

0.76 

50 

0.28 

0.97 

50 

0.38 

1.07 

50 

0.38 

1.14 

50 

Produce  technical  drawings 

0.28 

0.90 

50 

0.28 

0.97 

50 

0.30 

1.02 

50 

0.28 

0.97 

50 

Receive  clients,  patients,  guests 

0.24 

0.82 

50 

0.28 

1.01 

50 

0.32 

1.04 

50 

0.32 

1.04 

50 

Analyze  electronic  signals 

0.26 

0.72 

50 

0.28 

0.81 

50 

0.38 

0.99 

50 

0.30 

0.84 

50 

Load  1  iMilosd  field  arti  Her y/ tank  guns 

0.22 

0.62 

50 

0.27 

0.97 

49 

0.29 

0.98 

49 

0.29 

0.98 

49 

Construct  wooden  bldgs  1  other  struct 

0.18 

0.44 

50 

0.26 

0.85 

50 

0.24 

0.85 

50 

0.26 

0.85 

50 

Translate  foreign  languages 

0.24 

0.82 

50 

0.22 

0.74 

50 

0.32 

0.96 

SO 

0.30 

0.89 

50 

Fire  heavy  direct  fire  weapons 

0.10 

0.42 

50 

0.22 

0.89 

50 

0.14 

0.70 

50 

0.16 

0.79 

50 

Prepare  heavy  weapons  for  tactical  use 

0.14 

0.64 

50 

0.22 

0.89 

50 

0.20 

0.81 

50 

0.22 

0.89 

50 

Assemble  steel  structures 

0.20 

0.57 

50 

0.20 

0.73 

50 

0.20 

0.73 

50 

0.24 

0.74 

50 

Provide  medical  or  dental  treatment 

0.16 

0.74 

50 

0.20 

0.90 

50 

0.22 

0.91 

50 

0.22 

0.91 

50 

Fire  indirect  fire  weapons 

0.08 

0.34 

49 

0.16 

0.72 

49 

0.16 

0.72 

49 

0.16 

0.72 

49 

Operate  boats 

0.06 

0.31 

50 

0.16 

0.79 

SO 

0.16 

0.79 

50 

0.16 

0.79 

50 

Construct  masonry  bldgs  t  structures 

0.06 

0.31 

50 

0.16 

0.79 

50 

0.16 

0.79 

50 

0.16 

0.79 

50 

Operate  radar 

0.12 

0.48 

50 

0.16 

0.68 

50 

0.16 

0.68 

SO 

0.16 

0.68 

50 

Perform  medical  laboratory  procedures 

0.16 

0.82 

50 

0.16 

0.82 

50 

0.16 

0.82 

SO 

0.16 

0.82 

50 

Control  air  traffic 

0.04 

0.20 

50 

0.14 

0.70 

50 

0.14 

0.70 

50 

0.14 

0.70 

SO 

Prepare  parachutes 

0.06 

0.42 

50 

0.08 

0.57 

50 

0.08 

0,57 

50 

0.08 

0.57 

50 

Select, layout, tclean  mad/dental  equip 

0.06 

0.31 

50 

0.08 

0.57 

SO 

0.08 

0.57 

50 

0.08 

0.57 

50 

E-45 


TASK  CATEGORY 

Frequency  and  Iirportance  Ratings  Descriptive  Statistics 

Ordered  by  Core  Technical  Rating  Means 

MOS=71L 


Frequency 

Core  Technical 

Gen.  Soldier! 

ng 

Overal 

1  Job 

Task  Categories 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

MEAN 

STD 

N 

Type 

4.77 

0.61 

52 

4.75 

0.56 

51 

1.94 

1.67 

52 

4.19 

1.14 

52 

Record,  file,  and  dispatch  information 

4.46 

0.83 

52 

4.25 

0.84 

52 

2.15 

1.61 

52 

3.92 

1.01 

52 

Prepare  technical  forms  and  docunents 

4.15 

1.13 

52 

4.02 

1.08 

52 

2.00 

1.57 

52 

3.69 

1.12 

51 

Read  tech/field  manuals,  reg  etc 

3.52 

1.35 

52 

3.63 

1.34 

52 

3.48 

1.26 

52 

3.65 

1.25 

52 

Write  docunents  and  correspondence 

2.75 

1.67 

52 

2.90 

1.64 

51 

1.73 

1.57 

52 

2.84 

1.65 

51 

Operate  conputer  hardware 

3.0B 

1.57 

52 

2.88 

1.66 

51 

1.59 

1.63 

51 

2.88 

1.68 

52 

Give  directions  and  instructions 

3.00 

1.31 

52 

2.82 

1.29 

51 

2.77 

1.48 

52 

3.19 

1.17 

52 

Reproduce  printed  material 

3.04 

1.57 

52 

2.69 

1.77 

51 

1.33 

1.46 

52 

2.63 

1.67 

52 

Receive, store, (issue  sipp,  equip,  etc 

2.83 

1.52 

52 

2.56 

1.41 

52 

2.08 

1.48 

52 

2.71 

1.26 

52 

Act  as  a  model 

2.52 

1.85 

52 

2.44 

1.88 

50 

2.87 

1.87 

52 

2.98 

1.82 

52 

Personnel  Actninstration 

2.38 

1.90 

52 

2.41 

1.99 

51 

1.98 

1.95 

52 

2.38 

1.89 

52 

Comnuiicate 

2.50 

1.93 

52 

2.37 

1.94 

51 

2.63 

1.99 

52 

2.65 

1.96 

52 

Operate  electronic  equipment 

1.92 

1.77 

52 

1.94 

1.93 

51 

1.48 

1.64 

52 

2.10 

1.81 

52 

Lead 

1.94 

1.81 

52 

1.94 

1.89 

SO 

2.47 

2.03 

51 

2.37 

2.05 

52 

Cotrisel 

1.85 

1.73 

52 

1.75 

1.81 

51 

2.17 

1.97 

52 

2.15 

1.99 

52 

Train 

1.62 

1.72 

52 

1.73 

1.95 

51 

2.10 

2.01 

52 

2.12 

2.01 

52 

Order  equipment  and  supplies 

1.88 

1.34 

52 

1.51 

1.24 

51 

1.35 

1.36 

52 

1.71 

1.32 

52 

Monitor/ inspect 

1.38 

1.66 

52 

1.39 

1.78 

51 

1.85 

1.96 

52 

1.75 

1.86 

52 

Use  maps 

1.87 

1.16 

52 

1.37 

1.40 

51 

3.52 

1.38 

52 

2.88 

1.46 

52 

Give  short  oral  reports 

1.37 

1.09 

52 

1.37 

1.28 

51 

2.38 

1.60 

52 

2.19 

1.53 

52 

Interview 

1.31 

1.54 

52 

1.35 

1.57 

52 

1.31 

1.52 

52 

1.48 

1.54 

52 

Operate  wheeled  vehicles 

1.79 

1.14 

52 

1.31 

1.39 

51 

2.44 

1.50 

52 

2.27 

1.50 

52 

Receive  clients,  patients,  guests 

1.54 

1.72 

52 

1.25 

1.53 

52 

0.79 

1.43 

52 

1.35 

1.57 

52 

Protect  against  NBC  hazards 

2.04 

1.07 

52 

1.24 

1.39 

49 

4.02 

1.13 

52 

3.37 

1.36 

52 

Survive  in  the  field 

1.77 

1.02 

52 

1.24 

1.51 

50 

3.98 

1.20 

52 

3.35 

1.33 

52 

Give  first  aid 

1.88 

1.00 

52 

1.22 

1.39 

50 

4.17 

0.81 

52 

3.54 

1.09 

52 

Perf  operator  chks  (  services  on  weapons 

1.77 

1.23 

52 

1.22 

1.19 

51 

3.47 

1.55 

51 

2.96 

1.48 

51 

Perf  operator  maint  chks  and  services 

1.56 

1.26 

52 

1.18 

1.32 

SO 

2.37 

1.70 

51 

2.12 

1.60 

51 

Fire  individual  weapons 

1.98 

1.02 

52 

1.18 

1.40 

50 

4.31 

0.90 

52 

3.56 

1.19 

52 

Provide  progranming  (  OP  support 

1.15 

1.70 

52 

1.18 

1.58 

51 

0.69 

1.18 

52 

1.12 

1.44 

52 

Write  and  deliver  presentations 

1.08 

1.33 

52 

1.08 

1.41 

51 

0.86 

1.33 

51 

1.18 

1.38 

51 

Move  and  react  in  the  field 

1.60 

1.14 

52 

0.98 

1.24 

50 

3.56 

1.66 

52 

3.06 

1.55 

52 

Customs  and  laws  of  war 

1.38 

1.07 

52 

0.90 

1.30 

50 

3.10 

1.50 

52 

2.54 

1.46 

52 

Provide  couiseling 

0.88 

1.35 

52 

0.86 

1.40 

51 

1.04 

1.53 

52 

1.17 

1.61 

52 

Send  and  receive  radio  aiessages 

1.22 

1.01 

51 

0.84 

1.13 

50 

2.41 

1.80 

51 

1.92 

1.55 

51 

Conpute  statistics/other  math 

0.71 

1.07 

52 

0.83 

1.28 

52 

0.50 

0.96 

52 

0.85 

1.26 

52 

Navigate 

1.40 

1.05 

52 

0.82 

1.13 

49 

3.25 

1.71 

51 

2.71 

1.50 

51 

Operate  gas  (  electric  powered  equip 

0.92 

1.03 

52 

0.78 

1.14 

51 

1.31 

1.44 

52 

1.12 

1.29 

52 

Install  electronic  components 

0.85 

1.16 

52 

0.75 

1.14 

52 

1.10 

1.42 

52 

1.08 

1.41 

52 

Pack  and  load  materials 

1.00 

1.17 

52 

0.67 

1.02 

52 

1.40 

1.52 

52 

1.27 

1.42 

52 

Use  hand  grenades 

1.13 

0.91 

52 

0.64 

1.01 

50 

2.65 

1.67 

52 

2.25 

1.60 

52 

Use  hand  and  ana  signals 

1.06 

0.92 

52 

0.63 

0.87 

51 

2.13 

1.73 

52 

1.63 

1.39 

52 

Detect  and  identify  targets 

1.06 

0.98 

52 

0.60 

0.95 

50 

2.37 

1.84 

52 

1.94 

1.60 

52 

Place  C  camoufl  tactical  equip  and  mat 

0.92 

1.08 

52 

0.57 

1.19 

51 

1.84 

1.94 

51 

1.51 

1.74 

51 

Use  audiovisusl  aquipnent 

0.73 

1.12 

52 

0.55 

1.03 

51 

0.62 

1.07 

52 

0.79 

1.18 

52 

Control  money 

0.50 

0.85 

52 

0.55 

1.19 

51 

0.50 

1.00 

52 

0.63 

1.24 

52 

Decode  data 

0.58 

0.72 

52 

0.48 

0.96 

52 

1.23 

1.54 

52 

1.13 

1.43 

52 

Control  individuals  and  crowds 

0.98 

1.04 

52 

0.45 

0.90 

51 

1.96 

1.84 

52 

1.65 

1.64 

52 

J 
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TASK  CATEGORY  (Contd.) 


Frequency  and  Iinportance  Ratings  Descriptive  Statistics 
Ordered  by  Core  Technical  Rating  Means 

MOS=71L 

Frequency 

Task  Categories 

MEAN 

STD 

Install  wire  and  cables 

0.46 

0.83 

Sketch  maps,  overlays,  or  range  cards 

0.83 

0.98 

Inspect  electrical  systens 

0.40 

0.85 

Plan  operations 

0.44 

1.06 

Repair  mechanical  system 

0.21 

0.46 

Direct/ lead  team 

0.46 

0.73 

Engage  in  hand-to-hand  combat 

0.69 

1.02 

Troubleshoot  mechanical  system 

0.31 

0.67 

Repair  electrical  system 

0.27 

0.66 

Paint 

0.87 

0.97 

Inspect  electronic  system 

0.27 

0.72 

Troubleshoot  weapons 

0.21 

0.54 

Repair  weapons 

0.25 

0.59 

Repair  electronic  components 

0.29 

0.80 

Draw  illustrations 

0.17 

0.51 

Plan  placement /use  of  tactical  equip 

0.27 

0.53 

Translate  foreign  languages 

0.19 

0.53 

Estimate  time  t  cost  of  maint  ops 

0,27 

0.60 

Draw  maps  and  overlays 

0.19 

0.53 

Handle  demolitions  or  mines 

0.33 

0.68 

Assemble  steel  structures 

0.12 

0.43 

Conduct  land  surveys 

0.29 

0.67 

Operate  lifting, loadir>g, (grading  equip 

0.08 

0.34 

Prepare  equip  and  supplies  for  air  drop 

0.12 

0.32 

Provide  medical  or  dental  treatment 

0.08 

0.33 

Determine  firing  data- indirect  weapons 

0.10 

0.41 

Operate  track  vehicles 

0.15 

0.46 

Produce  technical  drawings 

0.13 

0.53 

Prepare  parachutes 

0.08 

0.33 

Repair  metal 

0.12 

0.32 

Construct  wooden  bldgs  (  other  struct 

0.08 

0.27 

Cook 

0.10 

0.36 

Fire  indirect  fire  weapons 

0.04 

0.28 

Repair  plastic  and  fiberglass 

0.08 

0.27 

Install  pipe  assemblies 

0.04 

0,19 

Operate  power  excavating  equipment 

0.02 

0.14 

Analyse  weather  conditiona 

0.06 

0.31 

Analyze  intelligarKe  data 

0.06 

0.24 

Operate  boats 

0.00 

0.00 

Construct  mesonry  bldgs  C  structures 

0.02 

0.14 

Select, layout, (clean  med/dcntal  equip 

0.04 

0.28 

Operate  radar 

0.02 

0.14 

Perform  medical  laboratory  procedures 

0.00 

0.00 

Analyze  electronic  signals 

0.00 

0.00 

Control  air  traffic 

0.00 

0.00 

Load  (  unload  field  artillery/tank  gm 

0.02 

0.14 

Fire  heavy  direct  fire  weapons 

0.00 

0.00 

Prepare  heavy  weapons  for  tactical  use 

0.00 

0.00 

Core  Technical 

Gen.  Soldieri 

no 

Overall  Job 

N 

MEAN 

STD 

N 

MEAN 

STO 

N 

MEAN 

STO 

N 

52 

0.43 

0.88 

51 

0.71 

1.29 

52 

0.63 

1.09 

52 

52 

0.43 

0.83 

51 

1.44 

1.66 

52 

1.17 

1.40 

52 

52 

0.41 

0.96 

51 

0.41 

1.02 

51 

0.53 

1.16 

51 

52 

0.41 

1.08 

51 

0.63 

1.31 

52 

0.60 

1.29 

52 

52 

0.37 

0.86 

52 

0,37 

0.86 

52 

0.35 

0.81 

52 

52 

0.35 

0.71 

52 

1.04 

1.55 

52 

0.90 

1.45 

52 

52 

0.29 

0.67 

51 

1.52 

1.89 

52 

1.25 

1.69 

52 

52 

0.27 

0.66 

52 

0.54 

1.15 

52 

0.46 

1.00 

52 

52 

0.23 

0.61 

52 

0.25 

0.71 

52 

0.25 

0.65 

52 

52 

0.23 

0.51 

52 

0.60 

0.75 

52 

0.54 

0.83 

52 

52 

0.21 

0.64 

52 

0.33 

0.92 

52 

0.35 

0.88 

52 

52 

0,19 

0.53 

52 

0.48 

1.20 

52 

0.44 

1.11 

52 

52 

0.17 

0.51 

52 

0.52 

1.20 

52 

0.42 

1.04 

52 

52 

0.17 

0.51 

52 

0.21 

0,70 

52 

0,25 

0,71 

52 

52 

0.17 

0.55 

52 

0.25 

0.76 

52 

0.17 

0,51 

52 

52 

0.16 

0.46 

51 

0.60 

1.27 

52 

0.44 

0.98 

52 

52 

0.16 

0.73 

51 

0.21 

0.64 

52 

0.25 

0.68 

52 

52 

0.14 

0.53 

51 

0.38 

1.03 

52 

0.35 

0.95 

52 

52 

0.13 

0.53 

52 

0.25 

0.79 

52 

0,19 

0.63 

52 

52 

0,12 

0.38 

52 

0.69 

1.42 

52 

0.52 

1.18 

52 

52 

0.10 

0.50 

51 

0,19 

0.72 

52 

0.17 

0.65 

52 

52 

0.10 

0.36 

51 

0.55 

1.25 

51 

0,39 

0.98 

51 

51 

0.08 

0.44 

51 

0.10 

0.57 

51 

0.10 

0.57 

51 

52 

0.08 

0.27 

52 

0.19 

0.60 

52 

0.13 

0.40 

52 

52 

0.08 

0.39 

52 

0.23 

0.94 

52 

0.19 

0.79 

52 

52 

0.08 

0.39 

52 

0.12 

0.51 

52 

0.08 

0.39 

52 

52 

0.06 

0.24 

51 

0.19 

0.66 

52 

0.17 

0.62 

52 

52 

0.06 

0.31 

51 

0.21 

0.78 

52 

0.13 

0.53 

52 

52 

0.06 

0.31 

52 

0.17 

0.81 

52 

0.13 

0.71 

52 

52 

0.04 

0.20 

51 

0.08 

0.27 

51 

0.08 

0.27 

51 

52 

0.04 

0.20 

51 

0.12 

0.43 

52 

0.12 

0.43 

52 

52 

0.04 

0.19 

52 

0.06 

0.24 

52 

0.04 

0.19 

52 

52 

0.04 

0.28 

52 

0.06 

0.42 

52 

0.06 

0.42 

52 

52 

0.02 

0.14 

51 

0.08 

0.33 

52 

0.08 

0.33 

52 

52 

0.02 

0.14 

51 

0.02 

0.14 

51 

0.02 

0.14 

51 

52 

0.02 

0.14 

52 

0.06 

0.42 

52 

0.06 

0.42 

52 

52 

0.02 

0.14 

52 

0.08 

0.44 

52 

0.06 

0.42 

52 

52 

0.02 

0.14 

52 

0.10 

0.41 

52 

0.06 

0.31 

52 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

52 

0.00 

0.00 

51 

0.04 

0.28 

52 

0.04 

0.28 

52 

52 

0.00 

0.00 

51 

0.10 

0.69 

52 

0.10 

0.69 

52 

52 

0.00 

0.00 

51 

0.08 

0.55 

52 

0.08 

0.55 

52 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

52 

0.00 

0.00 

52 

0.06 

0.42 

52 

0.06 

0.42 

52 

52 

o.te 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 

0.00 

0.00 

52 
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JOB  ACTIVITY 

Frequefxry  and  Importance  Ratings  Descriptive  Statistics 

Ordered  by  Core  Technical  Rating  Means 

HOS=11B 

Frequency  Core  Technical  Gen.  Soldiering  Overall  Job 

Job  Activities _ MEAN  STD  M  MEAN  STD  W  MEAN  STD  N  MEAN  STD  M 


Aim: stationary  target 

4.10 

1.00 

Work  in  a  team 

4.18 

0.90 

Aim: moving  target 

3.66 

1.35 

Run  short  distances 

4.22 

1.05 

Walk  long  distances 

4.10 

1.12 

Work  under  adverse  conditions 

4.07 

1.11 

Follow  oral  directions 

3.86 

1.45 

Work  long  hours 

4.21 

0.86 

Judge  location 

3.00 

1.41 

Follow  written  directions 

3.41 

1.31 

Judge  size  and  distance 

3.00 

1.37 

Push,  pull,  lift  heavy  weights 

3.48 

1.33 

Throw  objects 

3.23 

1.36 

Support/advise  peers 

2.94 

1.46 

Relay  oral  instructions 

3.07 

1.51 

Lead  a  team 

2.28 

1.47 

Coach  subordinates 

2.67 

1.57 

Coach  peers 

2.70 

1.44 

Support/advise  SLbordi nates 

2.60 

1.58 

Make  coordinated  movements 

2.69 

1.73 

Recall  verbal  information 

2.70 

1.63 

Make  oral  reports  (to  irdividuals) 

2.70 

1.54 

Drive  tracked  vehicle 

2.54 

1.86 

Record  information 

2.49 

1.41 

Judge  paths  of  moving  objects 

2.20 

1.57 

Drive  light  wheeled  vehicle 

2.20 

1.46 

Monitor/ interpret  verbal  messages 

2.13 

1.77 

Operate  hand-held  tools 

2.38 

1.56 

Solve  tactical  maneuver  problems 

1.75 

1.64 

Adjust  control  device  using  one  linb 

2.01 

1.76 

Write  brief  messages 

1.95 

1.55 

Solve  leadership  problem 

1.65 

1.63 

Monitor/ interpret  figural  information 

1.64 

1.41 

Adj  control  device  using  mult  linba 

1.89 

1.82 

Recall  figural  information 

1.68 

1.50 

Solve  coimnication  problem 

1.58 

1.48 

Recall  nunerical  informtion 

1.74 

1.58 

Solve  mechanical  system  problem 

1.51 

1.49 

Make  oral  reports  (to  groups) 

1.51 

1.41 

Drive  heavy  wheeled  vehicle 

1.52 

1.61 

Noni tor/interpret  numrical  informtion 

1.U 

1.49 

Interview 

1.38 

1.51 

Sort,  fold,  feed  by  hand 

1.48 

1.72 

Solve  adainistrative  problem 

1.00 

1.29 

Solve  electrical  system  problem 

0.97 

1.35 

Solve  medical  problem 

0.85 

1.22 

Solve  logistical  problem 

0.80 

1.22 

Write  longer  reports 

0.64 

1.01 

Operate  hand-held  power  equipment 

0.86 

1.25 

Operate  precision  hand-held  equipment 

0.69 

1.43 

Operate  larger  power  equipment 

0.59 

1.27 

Operate  full  keyboard 

0.53 

0.91 

Operate  nuimric  keyboard 

0.32 

0.74 

87 

4.56 

0.85 

86 

4.15 

1.13 

86 

4.40 

0.95 

86 

88 

4.37 

0.98 

87 

4.22 

1.11 

87 

4.46 

0.83 

87 

87 

4.32 

1.12 

87 

3.91 

1.30 

87 

4.16 

1.16 

87 

87 

4.24 

1.03 

86 

3.83 

1.30 

86 

4.13 

1.03 

86 

87 

4.17 

1.21 

86 

3.65 

1.45 

86 

4.01 

1.16 

86 

86 

3.95 

1.34 

85 

3.56 

1.47 

85 

3.99 

1.17 

85 

88 

3.84 

1.49 

86 

3.84 

1.44 

85 

4.00 

1.39 

85 

87 

3.74 

1.36 

86 

3.36 

1.46 

86 

3.64 

1.32 

86 

88 

3.71 

1.56 

86 

3.08 

1.54 

86 

3.35 

1.49 

86 

88 

3.65 

1.36 

86 

3.72 

1.35 

85 

3.84 

1.38 

85 

88 

3.55 

1.42 

86 

2.94 

1.34 

86 

3.26 

1.29 

86 

87 

3.51 

1.47 

86 

3.09 

1.47 

86 

3.37 

1.39 

86 

87 

3.45 

1.46 

86 

3.00 

1.53 

86 

3.22 

1.45 

86 

88 

3.26 

1.51 

87 

3.20 

1.43 

87 

3.32 

1.43 

87 

88 

3.26 

1.57 

87 

3.16 

1.55 

87 

3.31 

1.50 

87 

88 

3.07 

1.66 

87 

2.89 

1.60 

87 

3.14 

1.63 

87 

86 

2.99 

1.64 

84 

3.01 

1.65 

84 

3.15 

1.71 

84 

88 

2.97 

1.56 

87 

3.01 

1.56 

87 

3.08 

1.54 

87 

88 

2.94 

1.67 

87 

2.96 

1.67 

87 

3.03 

1.71 

87 

87 

2.92 

1.75 

86 

2.51 

1./5 

86 

2.71 

1.80 

86 

88 

2.84 

1.71 

86 

2.69 

1.59 

85 

2.78 

1.61 

85 

88 

2.76 

1.69 

87 

2.82 

1.65 

87 

2.84 

1.61 

87 

87 

2.68 

1.89 

87 

2.25 

1.70 

87 

2.59 

1.79 

87 

88 

2.67 

1.51 

87 

2.68 

1.57 

87 

2.80 

1.58 

87 

88 

2.65 

1.83 

86 

2.19 

1.61 

86 

2.38 

1.65 

86 

87 

2.46 

1.59 

87 

2.34 

1.51 

87 

2.47 

1.50 

87 

88 

2.38 

1.78 

86 

2.26 

1.75 

85 

2.32 

1.75 

85 

87 

2.30 

1.60 

86 

2.16 

1.47 

86 

2.29 

1.54 

86 

88 

2.27 

1.92 

86 

1.87 

1.74 

86 

2.15 

1.89 

86 

87 

2.09 

1.87 

87 

1.80 

1.73 

87 

2.09 

1.86 

87 

88 

2.06 

1.74 

87 

2.17 

1.78 

87 

2.22 

1.79 

87 

88 

2.01 

1.86 

86 

1.88 

1.81 

86 

2.07 

1.92 

86 

88 

2.00 

1.74 

86 

1.84 

1.62 

85 

1.82 

1.60 

85 

87 

1.95 

1.85 

86 

1.81 

1.77 

86 

1.94 

1.82 

86 

88 

1.88 

1.75 

86 

1.71 

1.58 

85 

1.88 

1.73 

85 

88 

1.87 

1.68 

87 

1.84 

1.66 

87 

1.91 

1.73 

87 

88 

1.79 

1.65 

86 

1.85 

1.63 

85 

1.86 

1.66 

85 

88 

1.79 

1.77 

86 

1.57 

1.59 

86 

1.69 

1.66 

86 

88 

1.74 

1.74 

87 

1.82 

1.73 

87 

1.91 

1.74 

87 

87 

1.67 

1.75 

87 

1.63 

1.66 

87 

1.68 

1.68 

87 

88 

1.57 

1.55 

86 

1.54 

1.52 

85 

1.59 

1.60 

85 

88 

1.54 

1.70 

87 

1.52 

1.67 

87 

1.54 

1.67 

87 

87 

1.47 

1.82 

87 

1.41 

1.73 

87 

1.47 

1.79 

87 

88 

1.23 

1.61 

86 

1.28 

1.67 

86 

1.36 

1.71 

86 

88 

1.08 

1.53 

87 

1.01 

1.42 

87 

1.01 

1.40 

87 

88 

0.98 

1.46 

86 

0.93 

1.43 

86 

1.02 

1.56 

86 

88 

0.94 

1.39 

86 

0.91 

1.34 

86 

0.93 

1.38 

86 

87 

0.88 

1.40 

86 

0.79 

1.29 

86 

0.84 

1.33 

86 

87 

0.88 

1.22 

86 

0.93 

1.28 

86 

0.88 

1.23 

86 

88 

0.72 

1.43 

88 

0.66 

1.35 

88 

0.72 

1.43 

88 

86 

0.60 

1.28 

86 

0.51 

1.17 

86 

0.57 

1.22 

86 

87 

0.46 

0.96 

87 

0.51 

0.99 

87 

0.51 

0.97 

87 

87 

0.34 

0.78 

87 

0.34 

0.79 

87 

0.38 

0.84 

87 
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Frequency  Core  Technical  Gen.  Soldiering  Overall  Job 


Job  Activities 

MEAN 

STD 

N 

MEAN 

STD 

N 

STD 

N 

MEAN 

STD 

N 

Operate  hand-held  tools 

4.18 

1.00 

50 

4.38 

0.88 

SO 

2.54 

1.34 

SO 

3.80 

1.20 

so 

Solve  mechanical  system  problems 

3.62 

1.07 

50 

4.12 

1.00 

SO 

2.36 

1.21 

SO 

3.50 

1.20 

so 

Operate  hand-held  power  equipment 

3.62 

1.18 

50 

4.04 

1.03 

SO 

2.34 

1.35 

SO 

3.44 

1.33 

50 

Drive  light  wheeled  vehicle 

3.94 

1.11 

50 

4.02 

1.19 

50 

3.16 

1.25 

50 

3.74 

1.12 

50 

Follow  written  directions 

3.88 

0.98 

50 

3.94 

1,11 

SO 

3.74 

1.03 

50 

3.94 

1,11 

SO 

Solve  electrical  system  problems 

3.22 

1.25 

50 

3.92 

1.26 

50 

2.10 

1.46 

50 

3.36 

1.26 

50 

Follow  oral  directions 

3.74 

1.07 

50 

3.70 

1.05 

SO 

3.72 

1.01 

50 

3.88 

0.98 

50 

Operate  precision  hand-held  equipment 

2.92 

1.52 

50 

3.52 

1.54 

SO 

2.18 

1.30 

50 

2.94 

1.46 

50 

Work  in  a  team 

3.32 

1.38 

50 

3.34 

1.32 

50 

3.70 

1.27 

50 

3.58 

1.21 

50 

Adj  control  device  using  mult  limbs 

3.02 

1.46 

50 

3.30 

1.56 

50 

2.24 

1.32 

49 

2.86 

1.51 

49 

Work  long  hours 

3.22 

1.15 

50 

3.28 

1.39 

50 

3.30 

1.37 

50 

3.38 

1.19 

50 

Push,  pull,  lift  heavy  weights 

3.42 

0.97 

50 

3.26 

1.31 

50 

3.20 

1.12 

50 

3.38 

1.01 

SO 

Drive  heavy  wheeled  vehicle 

2.80 

1.60 

50 

3.16 

1.71 

SO 

2.46 

1.45 

50 

2.94 

1.60 

50 

Work  under  adverse  conditions 

2.94 

1.39 

50 

3.14 

1.58 

so 

2.98 

1.49 

50 

3.14 

1.50 

50 

Adjust  control  device  using  one  linb 

2.88 

1.55 

50 

3.10 

1.59 

50 

2.08 

1.26 

50 

2.68 

1.56 

50 

Hake  coordinated  movements 

2.86 

1.28 

50 

3.10 

1.43 

50 

2.66 

1.27 

50 

2.98 

1.29 

50 

Judge  paths  of  moving  objects 

2.68 

1.02 

50 

2.90 

1.25 

50 

2.86 

1.01 

SO 

3.00 

1.05 

SO 

Record  information 

2.58 

1.36 

50 

2.82 

1.48 

49 

2.41 

1.26 

49 

2.67 

1.33 

49 

Relay  oral  instructions 

2.52 

1.25 

50 

2.70 

1.33 

SO 

2.74 

1.31 

50 

2.64 

1.24 

SO 

Recall  verbal  information 

2.58 

1.57 

50 

2.68 

1.66 

SO 

2.70 

1.67 

50 

2,74 

1.61 

50 

Hake  oral  reports  (to  individuals) 

2.38 

1.37 

SO 

2.54 

1.42 

50 

2.68 

1.32 

50 

2.66 

1.30 

50 

Judge  size  and  distance 

2.40 

1.26 

50 

2.52 

1.49 

50 

2.76 

1.33 

50 

2.76 

1.35 

50 

Support/ advise  subordinates 

2.26 

1.58 

SO 

2.50 

1.58 

50 

2.90 

1.57 

50 

2.74 

1.52 

50 

Support/advise  peers 

2.44 

1.49 

SO 

2.48 

1.54 

50 

2.84 

1.61 

50 

2.68 

1.52 

50 

Judge  location 

2,45 

0.82 

49 

2.45 

1.21 

49 

3.10 

1,10 

49 

2.98 

0.99 

49 

Coach  subordinates 

2.12 

1.62 

SO 

2.38 

1.65 

50 

2.66 

1.77 

50 

2.42 

1.63 

50 

Recall  nunerical  information 

2.06 

1.35 

SO 

2.36 

1.56 

SO 

2.26 

1.51 

50 

2.20 

1.47 

50 

Lead  a  team 

2.08 

1.55 

SO 

2.34 

1.64 

50 

2.76 

1.62 

50 

2.64 

1.56 

50 

Rm  short  distances 

3.54 

1.07 

SO 

2.32 

1.77 

50 

3.88 

1.06 

50 

3.52 

1.16 

50 

Coach  peers 

2.10 

1.39 

SO 

2.30 

1.49 

50 

2.56 

1.58 

50 

2.36 

1.47 

50 

Honi tor/interpret  figural  information 

1.74 

1.66 

SO 

2.12 

1.88 

50 

1.82 

1.56 

so 

1.94 

1.72 

SO 

Operate  larger  power  equipment 

1.82 

1.52 

SO 

2.12 

1.65 

50 

1.26 

1.37 

50 

2.00 

1.77 

50 

Recall  figural  information 

1.68 

1.38 

SO 

2.08 

1.66 

50 

1.92 

1.43 

50 

1.96 

1.55 

50 

Walk  long  distarfces 

2.38 

1.24 

50 

2.04 

1.63 

50 

2.98 

1.45 

50 

2.82 

1.37 

SO 

Write  brief  messages 

1.82 

1.38 

SO 

1.94 

1.41 

50 

1.90 

1.28 

50 

1.96 

1.32 

50 

Solve  logistical  problvM 

1.56 

1.50 

50 

1.88 

1.67 

50 

1.48 

1,49 

50 

1.74 

1.58 

50 

Honi tor/ interpret  nunerical  information 

1.48 

1.25 

SO 

1.78 

1.5* 

50 

1.66 

1.44 

SO 

1.72 

1.49 

50 

Aim:stationary  target 

2.26 

1.21 

50 

1.78 

1.71 

50 

3.50 

1.52 

50 

3.00 

1.36 

50 

Drive  tracked  vehicle 

1.36 

1.35 

SO 

1.76 

1.70 

50 

1.56 

1.58 

50 

1.64 

1.56 

50 

Sort,  fold,  feed  by  hand 

1.64 

1.59 

50 

1.76 

1.64 

SO 

1.52 

1.54 

50 

1.70 

1.61 

SO 

Honi tor/ interpret  verbal  messages 

1.54 

1,59 

50 

1.74 

1,79 

50 

1.76 

1.78 

so 

1.74 

1.77 

so 

Aim:moving  target 

1,70 

1.37 

50 

1.72 

1.76 

50 

3.00 

1.82 

so 

2.56 

1.58 

so 

Make  oral  reports  (to  groups) 

1.42 

1.18 

50 

1.64 

1.35 

50 

1.88 

1.4S 

SO 

1.68 

1.35 

50 

Solve  leadership  problems 

1.36 

1.59 

■  50 

1,4* 

1.64 

50 

1.76 

1.72 

so 

1.62 

1.60 

50 

Solve  tactical  mansuver  problems 

1.20 

1,18 

50 

1.26 

1.45 

50 

1.82 

1.77 

50 

1.60 

1.58 

SO 

Solve  adninistrative  problems 

1.32 

1.53 

50 

1.26 

1.51 

SO 

1.30 

1.49 

SO 

1.34 

1.52 

SO 

Interview 

1.08 

1.37 

50 

1.24 

1.52 

SO 

1.18 

1.37 

SO 

1.22 

1.50 

50 

Solve  comnunication  problems 

1.10 

1.16 

50 

1.24 

1.36 

50 

1.54 

1.58 

SO 

1.40 

1.43 

50 

Throw  objects 

1.48 

1.37 

50 

1.12 

1.30 

SO 

1.86 

1.60 

SO 

1.68 

1,50 

50 

Operate  full  keyboard 

0.76 

0.94 

50 

0.92 

1.1* 

50 

0.84 

1.22 

SO 

0.94 

1.22 

50 

Operate  runtric  keyboard 

0.56 

0.84 

50 

0.72 

1.11 

SO 

0.S6 

0.99 

SO 

0.70 

1.13 

SO 

Write  longer  reports 

0.58 

0,97 

50 

0.70 

1.13 

SO 

0.80 

1.23 

SO 

0.70 

1.16 

50 

Solve  medical  problems 

0.68 

1.15 

50 

0.66 

1.19 

SO 

1.00 

1.56 

SO 

0.90 

1.40 

50 
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Job  Activities 

Frequency 
MEAN  STD 

N 

Core 

MEAN 

Technical 

STD  M 

Gen. 

MEAN 

Soldiering 

STD  N 

Overall  Job 

MEAN  STD 

N 

Operate  full  keyboard 

4.58 

0.78 

52 

4.54 

0.93 

50 

2.22 

1.74 

49 

4.02 

1.22 

50 

Follow  written  directions 

4.12 

0.94 

52 

4.41 

0.90 

51 

4.10 

1.15 

51 

4.37 

0.85 

51 

Follow  oral  directions 

3.90 

1.29 

52 

4.20 

1.11 

51 

4.00 

1.30 

51 

4.20 

1.10 

51 

Record  information 

3.52 

1.43 

52 

3.62 

1.30 

52 

2.73 

1.56 

52 

3.42 

1,35 

52 

Solve  adninistrative  problems 

3.00 

1.41 

52 

3.43 

1.50 

51 

2.37 

1.67 

51 

3.20 

1.47 

51 

Recall  verbal  information 

2.96 

1.30 

52 

3.24 

1.27 

51 

3.10 

1.32 

51 

3.22 

1.30 

51 

Write  brief  messages 

3.04 

1.63 

52 

3.21 

1.39 

52 

2.56 

1.55 

52 

3.13 

1.39 

52 

Relay  oral  instructions 

2.79 

1.32 

52 

3.18 

1.20 

51 

3.24 

1.41 

51 

3.31 

1.24 

51 

Work  in  a  team 

2.94 

1.36 

52 

3.08 

1.38 

52 

3.73 

1.29 

52 

3.52 

1.21 

52 

Support/advise  peers 

2.65 

1.27 

52 

2.88 

1.31 

51 

3.33 

1.16 

51 

3.24 

1.12 

51 

Monitor/ interpret  verbal  messages 

2.54 

1.42 

52 

2.88 

1.53 

51 

2.53 

1.47 

51 

2.80 

1.51 

51 

Coach  peers 

2.38 

1.30 

52 

2.59 

1.30 

51 

2.76 

1.42 

51 

2.69 

1.30 

51 

Hake  oral  reports  (to  individuals) 

2.62 

1.47 

52 

2.58 

1.47 

52 

2.67 

1.41 

52 

2.75 

1.43 

52 

Sort,  fold,  feed  by  hand 

2.58 

1.45 

52 

2.55 

1.55 

51 

1.63 

1.55 

51 

2.45 

1.45 

51 

Support/ advise  sibordinates 

2.17 

1.69 

52 

2.45 

1.76 

51 

2.65 

1.80 

51 

2.57 

1.69 

51 

Coach  subordinates 

2.12 

1.69 

52 

2.41 

1.73 

51 

2.55 

1.84 

51 

2.53 

1.75 

51 

Operate  runeric  keyboard 

2.16 

1.72 

51 

2.29 

1.79 

51 

1.22 

1.52 

50 

2.18 

1.75 

51 

Recall  miner i cal  information 

1.98 

1.31 

52 

2.28 

1.53 

50 

2.04 

1.50 

50 

2.36 

1.50 

50 

Monitor/ interpret  runerical  information 

1.83 

1.41 

52 

2.08 

1.56 

52 

1.67 

1.46 

52 

2.02 

1.54 

52 

Work  long  hours 

2.31 

1.25 

52 

1.94 

1.45 

51 

2.86 

1.46 

51 

2.63 

1.36 

51 

Interview 

1.73 

1.56 

52 

1.92 

1.62 

52 

1.56 

1.61 

52 

1.87 

1.60 

52 

Lead  a  team 

1.67 

1.59 

52 

1.80 

1.70 

51 

2.39 

1.78 

51 

2.37 

1.71 

51 

Make  oral  reports  (to  groups) 

1.69 

1.50 

52 

1.73 

1.54 

52 

1.94 

1.66 

52 

2.10 

1.65 

52 

Solve  leadership  problems 

1.52 

1.55 

52 

1.73 

1.79 

51 

1.90 

1.85 

51 

1.92 

1.78 

51 

Solve  coniHLrication  problems 

1.52 

1.53 

52 

1.67 

1.70 

51 

1.75 

1.83 

51 

1.78 

1.74 

51 

Write  longer  reports 

1.44 

1.39 

52 

1.53 

1.36 

51 

1.16 

1.29 

51 

1.57 

1.42 

51 

Monitor/ interpret  figural  information 

1.25 

1.33 

52 

1.38 

1.36 

52 

1.56 

1.63 

52 

1.58 

1.55 

52 

Work  under  adverse  conditions 

1.40 

1.09 

52 

1.37 

1.40 

51 

2.57 

1.85 

51 

2.27 

1.72 

51 

Recall  figural  information 

1.15 

1.14 

52 

1.33 

1.38 

51 

1.80 

1.76 

51 

1.63 

1.48 

51 

Run  short  distances 

2.71 

1.51 

52 

1.29 

1.45 

51 

3.43 

1.39 

51 

2.90 

1.30 

51 

Hake  coordinated  movements 

1.33 

1.37 

52 

1.25 

1.48 

52 

2.06 

1.69 

52 

1.88 

1.59 

52 

Drive  light  wheeled  vehicle 

1.62 

1.21 

52 

1.15 

1.27 

52 

2.08 

1.61 

52 

1.75 

1.53 

52 

Adj  control  device  using  mult  linta 

1.00 

1.34 

51 

1.04 

1.34 

51 

1.20 

1.54 

51 

1.24 

1.53 

51 

Judge  location 

1.33 

0.86 

52 

1.02 

1.15 

50 

2.96 

1.56 

51 

2.28 

1.50 

50 

Adjust  control  device  using  one  lint) 

1.00 

1.22 

51 

0.96 

1.22 

51 

1.14 

1.40 

51 

1.20 

1.36 

51 

Judge  size  and  distance 

1.13 

1.07 

52 

0.92 

1.15 

51 

2.15 

1.78 

52 

1.67 

1.46 

52 

Walk  long  distances 

1.46 

0.85 

52 

0.88 

0.95 

49 

2.90 

1.55 

51 

2.33 

1.40 

51 

Push,  pull,  lift  heavy  weights 

1.46 

1.09 

52 

0.83 

0.96 

52 

2.17 

1.59 

52 

1.90 

1.35 

52 

Aim:stationary  target 

1.65 

1.25 

52 

0.75 

1.18 

51 

3.15 

1.83 

52 

2.58 

1.59 

52 

Judge  paths  of  moving  objects 

0.79 

0.94 

52 

0.69 

1.03 

51 

1,51 

1.69 

51 

1.16 

1.39 

51 

Solve  logistical  problems 

0.71 

1.05 

52 

0.67 

1.17 

52 

0.90 

1.36 

52 

0.75 

1.20 

52 

Solve  mechanical  system  problems 

0.75 

1.08 

52 

0.63 

0.99 

52 

0.83 

1.26 

52 

0.75 

1.12 

52 

Operate  hsnd>held  tools 

0.71 

0.89 

52 

0.56 

0.87 

52 

1.15 

1.38 

52 

0.92 

1.13 

52 

Aim: moving  target 

1.00 

1.25 

52 

0.49 

0.99 

51 

1.90 

1.99 

52 

1.69 

1.73 

52 

Solve  electrical  system  problems 

0.60 

1.05 

52 

0.48 

0.87 

52 

0.78 

1.32 

51 

0.61 

1.02 

51 

Throw  objects 

0.69 

0.90 

52 

0.37 

0.69 

52 

1.29 

1.60 

52 

1.13 

1.43 

52 

Solve  amdical  problama 

0.38 

0.95 

52 

0.31 

0.85 

52 

0.46 

1.09 

52 

0.44 

1.07 

52 

Operate  precision  hand-held  equipment 

0.27 

0.74 

52 

0.29 

0.87 

52 

0,37 

0.93 

52 

0.31 

0.78 

52 

Solve  tactical  maneuver  problems 

0.40 

0.75 

52 

0.25 

0.62 

52 

0.73 

1.34 

52 

0.52 

0.98 

52 

Operate  hand-held  power  equipment 

0.37 

0.77 

52 

O.S 

0.62 

52 

0.42 

0.82 

52 

0.31 

0.64 

52 

Drive  heavy  wheeled  vehicle 

0.48 

0.78 

52 

0.21 

0.54 

52 

0.65 

1.20 

52 

0.48 

1.00 

52 

Operate  larger  power  equipment 

0.27 

0.66 

52 

0.17 

0.51 

52 

0.27 

0.72 

52 

0.23 

0.65 

52 

Drive  tracked  vehicle 

0.19 

0.53 

52 

0.06 

0.31 

52 

0.21 

0.64 

52 

0.13 

0.44 

52 

APPENDIX  F 

Overall  Performance  Standard  Instrument 
DERIVING  AN  OVERALL  PERFORMANCE  STANDARD 


You  have  just  completed  several  exercises  designed  to  set  standards 
for  specific  aspects  of  performance.  In  this  last  exercise,  we 
want  your  judgment  about  a  soldier's  overall  performance.  Specifi¬ 
cally,  we  want  to  know  the  extent  to  which  substandard  performance 
in  one  area  might  be  balanced  by  performance  in  another  area  that 
is  well  above  the  standard  for  that  area. 

For  this  exercise,  assume  that  there  are  three  major  areas  of 
performance: 

•  Core  Technical  Proficiency  (performance  on  MOS-specific 
tasks) 

•  Effort  and  Leadership 

•  Personal  Discipline 

Assume  that  40  soldiers  were  assigned  performance  ratings  for  each 
of  the  three  areas.  We  want  you  to  tell  us  what  overall  rating 
you  think  should  be  given  to  each  of  these  soldiers.  In  the  Overall 
Rating  column,  put  a  U  for  Unacceptable,  an  M  for  Marginal,  an  A 
for  Acceptable,  and  an  O  for  Outstanding. 


